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The  constraint  on  oil  flow  from  the  Middle  East  as  a 
result  of  the  1973  war  and  the  increased  sophistication  of 
aircraft  weapons  systems  are  two  important  factors  which  have 
contributed  significantly  to  the  interest  in  visual  flight 
simulation  as  an  integral  part  of  military  flight  training. 
Costs  associated  with  these  factors,  such  as  procurement  and 
fuel  costs,  are  providing  pressure  to  the  military  establish¬ 
ment  to  improve  their  capability  to  provide  military  pilots 
with  visual  flight  simulation  systems  which  do  not  impair 
combat  effectiveness  or  aviation  safety.  This  thesis  describes 
the  results  of  flight  simulation  utilization  by  the  commercial 
airline  industry,  analyzes  the  effectiveness  realized  by  using 
flight  simulators  to  supplement  military  training  in  different 
aviation  environments,  and  outlines  methodologies  for  measur¬ 
ing  and  improving  the  cost-effectiveness  of  the  systems. 
Recommendations  for  careful  study  are  made  in  areas  that  would 
improve  military  utilization  of  flight  simulation. 
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X.  INTRODUCTION 

A.  PURPOSE 

According  to  LtGen.  William  J.  White,  Deputy  Chief  of 
Staff  for  Aviation,  United  States  Marine  Corps,  the  most 
critical  issues  facing  Marine  aviation  during  the  1980's  will 
be  procuring  sufficient  aircraft  to  meet  the  Soviet  threat, 
retaining  sufficient  pilots  to  man  the  squadrons,  and  preserv¬ 
ing  those  aircrews  and  airframes  through  safety  [Ref.  1] .  It 
is  anticipated  that  the  F/A-18  will  be  introduced  to  the 
fighter  community  in  1984  and  the  AV-8B  will  reach  the  attack 
squadrons  in  1986.  The  procurement  of  these  new  high  perfor¬ 
mance,  sophisticated  aircraft  supports  the  issue  of  meeting 
the  threat;  however,  pilot  retention  and  aviation  safety  are 
two  areas  where  continuous  studies  are  being  performed  to 
formulate  answers  that  will  satisfy  the  Department  of  Defense 
goals  of:  (1)  development  of  better  pilots  and  aircrew  mem¬ 
bers;  (2)  maintenance  and  improvement  in  combat  readiness; 

(3)  reduction  of  training  and  operating  costs;  and  (4)  con¬ 
servation  of  scarce  resources  such  as  energy,  weapons,  and 
ammunition  [Ref.  2]. 

Constraints  such  as  procurement  costs,  fuel  costs,  risks 
of  flying,  ecology,  and  training  effectiveness  interject  addi 
tional  problems  in  the  search  for  answers  to  the  issues. 
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An  area  of  study  that  is  being  strongly  considered  to  provide 
some  relief  from  the  constraints  is  military  use  of  flight 
simulators. 

The  world's  military  forces  will  spend  almost  $10  billion 
by  1987  on  development  and  procurement  of  all  types  of  simu¬ 
lators  and  trainers.  The  Navy  will  allocate  $1.7  billion  for 
flight  simulators  and  trainers.  Major  Navy  flight  simulator 
programs  include  the  AV-8B  estimated  at  $52  million  and  the 
F/A-18  with  a  projected  investment  of  $172  million  [Ref.  3]. 
The  simulators  will  be  state-of-the-art  incorporating  such 
characteristics  as  computer  generated  imagery  (CGI)  and  six 
degrees  of  freedom  motion  systems  to  provide  the  pilot  with 
the  most  realistic  simulation  possible,  resulting  in  more 
effective  training.  An  additional  powerful  side  effect  of 
these  simulator  procurements  is  the  savings  they  represent  in 
men,  fuel,  weapons,  and  supply  support  resulting  in  a  coeffi¬ 
cient  of  resource  savings.  The  value  of  this  coefficient  has 
been  estimated  to  vary  between  5  and  100,  depending  on  the 
type  of  military  mission  being  trained  in  the  simulator 
[Ref.  4], 

The  purpose  of  this  thesis  is  to  determine  whether  visual 
flight  simulators  for  military  aircraft  are  cost  effective. 

A  secondary  purpose  is  to  gather  data  supporting  the  training 
value  of  visual  flight  simulation  and  its  impact  on  cost 
effectiveness . 
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B.  RESEARCH  METHODOLOGY 

Information  gathering  for  this  thesis  includes  library 
research#  phone  conversations  with  personnel  from  Headquarters 
Marine  Corps,  Naval  Air  Systems  Command,  Naval  draining  Equip¬ 
ment  Center,  and  on-site  visits  to  a  major  airline  Plight 
Training  Center,  and  the  Human  Resources  Laboratory,  Williams 
AFB,  Arizona,  to  conduct  interviews  with  persons  involved  in 
both  cost  and  training  effectiveness  of  visual  flight 
simulators. 

C.  THESIS  ORGANIZATION 

Section  II  provides  a  background  for  flight  simulation 
including  historical  development,  a  description  of  simulation 
methodology  such  as  model  boards,  computer  animated  photographic 
terrain  view  (CAPTV) ,  and  CGI,  and  a  brief  description  of  three 
tactical  flight  simulators,  two  of  which  are  used  by  the  Navy 
and  one  by  the  Air  Force.  Section  III  examines  flight  simu¬ 
lator  utilization  within  commercial  aviation.  Section  IV 
presents  the  training  effectiveness  of  military  flight  simu¬ 
lators  to  include  methods  of  measurement,  analysis  of  training 
data,  and  characteristics  of  the  flight  simulator  program 
necessary  to  support  positive  training  ef feotiveness.  Section 
V  will  examine  the  cost  effectiveness  of  fD.ight  simulation, 
and  Section  VI  will  include  the  author's  conclusions  and 
recommendations. 
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II.  FLIGHT  SIMULATION  -  AN  OVERVIEW 


A.  HISTORICAL  DEVELOPMENT  OF  FLIGHT  SIMULATION 

The  historical  development  of:  flight  simulation  seems  to 
be  as  innovative  as  the  airplane;  itself.  The  motivational 
requirement  for  inventiveness  was  stimulated  by  the  hazards 
of  flying,  the  skill  required  to  pilot  the  airplane/  and  the 
need  for  a  training  aid  to  supplement  pilot  instruction.  The 
earliest  devices  appear  to  have  been  devised  around  1910  using 
actual  aircraft/  some  of  which  were  proposed  to  be  moved  at 
speed  in  the  air  supported  by  balloons  and  overhead  gantries. 

A  training  device  which  came  to  be  known  as  the  "penguin"  was 
also  developed  during  this  time  period.  It  was  a  stubbed¬ 
winged  aircraft  capable  of  moving  across  large  open  spaces 
but  incapable  of  leaving  the  ground  [Ref.  5], 

Prior  to  World  War  I,  special  training  apparatuses/  not 
based  upon  actual  aircraft/  were  being  developed  to  meet 
specific  needs  of  pilot  training.  For  example,  small  aircraft¬ 
like  devices  that  were  mounted  on  pivoted  universal  joints 
were  used  to  show  pilots  the  effect  of  prevailing  winds  on 
aerodynamic  control  surfaces.  During  the  war,  research 
attempts  were  made  to  use  simulators  for  aircrew  training. 

One  such  research  attempt  was  a  piece  of  equipment,  produced 
in  France  in  1917,  which  used  an  aircraft  fuselage  based  on 
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a  pivotal  mount  and  incorporated  compressed  air  which  produced 
variations  of  response  and  aerodynamic  feel  with  variances  in 
speed  [Ref.  6]. 

Zn  1924,  two  English  research  workers,  Reid  and  Burton, 
evaluated  the  importance  of  full  cockpit  simulation  by  meas¬ 
uring  responses  of  pilots  in  a  modified  aircraft  fuselage 
with  functioning  displays  and  controls.  It  was  concluded 
that  devices  which  required  pilots  to  make  responses  on  the 
ground  to  those  made  while  airborne  could  be  used  to:  (1) 
test  the  pilot's  ability  to  fly  and  land  successfully;  (2) 
assess  the  rate  of  acquisition  of  flying  skills;  (3)  train 
pilots  on  those  particular  motor  skills  necessary  for  aircraft 
controllability;  and  (4)  classify  subjects  for  different  forms 
of  flying  service  [Ref.  7], 

The  year  1929  proved  to  be  a  "banner  year"  in  flight  sim¬ 
ulator  development.  Rocder,  a  German  inventor,  proposed  an 
apparatus  for  instruction  in  the  navigation  of  vehicles  in 
free  space  utilizing  a  hydraulic  system  which  would  reproduce 
the  physical  movement  of  an  airship  not  unlike  the  motion  sys¬ 
tems  of  some  present-day  simulators.  The  first  Link  Trainer 
was  also  completed  in  1929  by  Edwin  Link  in  the  basement  of 
his  father's  piano  and  organ  factory  in  Binghamton,  New  York 
[Ref.  8].  The  drive  for  the  instruments  and  motion  system  of 
the  trainer  used  the  pneumatic  technology  of  organ  building. 
The  Second  World  War  provided  the  necessary  impetus  for  the 
development  of  the  Link  as  a  mass  produced  ground  trainer. 
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After  the  war,  development  of  computer  technology  accel¬ 
erated  the  design  of  flight  simulators.  First  in  analog  and 
now  in  digital  form,  the  modern  flight  simulator  uses  sophis¬ 
ticated  computing  techniques  to  animate  full  scale  represen¬ 
tations  of  the  operations],  environment  experienced  by  the 
pilot  during  flight. 

B.  VISUAL  FLIGHT  SIMULATION  METHODOLOGY 
1.  Model  Board  is 

The  first  method  described  by  the  author  for  visual 
flight  simulation  is  the  model  board.  As  a  reference,  the 
system  described  is  used  on  the  2F-87  Operational  Flight 
Trainer  (OFT)  for  the  P-3  aircraft. 

The  model  board  provides  the  pilot  with  a  realistic 
view  of  external  scenery  enabling  flight  crews  to  perform 
visual  take-off,  approach,  landing,  and  low-altitude  maneuver¬ 
ing  procedures  under  day,  dusk,  or  night  conditions.  The 
instructor  also  has  the  capability  of  selectively  varying 
visibility  and  cloud  effects. 

The  mf.in  system  includes  a  rigid  three  dimensional 
model  of  an  airfield  located  in  the  midst  of  a  landscape 
scaled  at  2000:1,  ei  closed  circuit  color  television  system, 
and  a  gantry-mounted  camera  which  moves  over  the  model  board 
to  simulate  height  and  movement  over  the  ground  in  response 
to  simulator  control  inputs.  A  special  optical  probe,  attached 
in  front  of  the  camera,  contains  servo-controlled  optical 
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systems  which  simulate  the  in-flight  pitch,  bank,  and  heading 
changes  [Ref.  9].  A  camera  trip  system  is  provid'd  to  prevent 
damage  to  the  optical  probe  due  to  contact  with  the  model 
surface.  Using  a  pressure-sensitive  switch  for  activation 
allows  the  camera  and  probe  to  retract  should  it  come  in  con¬ 
tact  with  the  model. 

Three  drive  systems,  driven  by  signals  originating 
in  the  OFT,  control  movement  of  the  camera  along  the  longitud¬ 
inal,  lateral,  and  height  axes  as  illustrated  in  Figure  II-l. 
Total  travel  of  the  longitudinal  (X  axis)  and  lateral  (Y  axis) 
drive  systems  are  approximately  37  feet  and  12  feet  respec  - 
tively  and  are  equivalent  to  a  maneuvering  area  of  12  by  4 
nautical  miles  over  the  2000:1  scale  model  board.  The  maximum 
travel  in  the  vertical  system  {Z  axis)  is  12  inches,  which  is 
equivalent  to  the  height  ox  2000  feet  [Ref.  10]. 

Detail  of  the  terrain  is  provided  by  a  pitch  mirror 
which  is  set  at  an  angle  of  45  degrees  to  the  optical  axis 
when  the  OFT  is  in  level  flight.  The  angle  of  the  mirror  can 
be  changed  +  12.25  degrees,  simulating  aircraft  movement. 
Outside  of  this  range  an  in-cloud  picture  is  presented  to  the 
pilot.  The  effects  of  bank  angle  are  simulated  by  using  a 
dove  prism  in  the  optical  probe,  and  the  nose  section,  includ¬ 
ing  the  pitch  mirror,  is  rotated  about  the  optical  path  to 
simulate  heading  change.  Focal  distance,  from  the  pilot's 
view,  is  continuously  updated,  resulting  in  a  signal  being 
applied  to  the  focus  drive  system.  The  optical  system,  as 
described,  is  depicted  in  Figure  II-2. 


The  mode]  board  is  illuminated  by  a  bank  of  350 
eight  foot  (244  cm)  fluorescent  tubes  located  approximately 
seven  feet  from  the  model  surface  [Ref.  11].  There  is  also 
lighting  on  the  camera  tower  to  eliminate  shadows  cast  by  the 
tower  structure  which  moves  on  a  track  between  the  model  board 
and  the  lighting  bank.  Control  of  the  light  banks  creates 
the  simulation  of.  day/  dusk,  and  night  conditions.  Small 
prisms  illuminated  from  behind  the  model  are  used  to  simulate 
the  airfield  lighting  system,  as  shown  in  Figure  II-3. 
Approach,  strobe,  visual  approach  slope  indicators,  touch-down 
zone  lighting,  runway  end  identification  lights,  runway,  and 
taxiway  lighting  are  included  within  this  system.  Realism 
of  the  night  scene  is  enhanced  even  more  by  lighting  elements 
incorporated  into  the  terrain  around  the  airfield  simulating 
lights  from  a  city. 

Figure  II-4  illustrates  the  positioning  of  the  pro¬ 
jector,  mirrors,  and  screen  necessary  for  the  display  system. 
The  high  brightness  of  the  display  allows  the  level  of  cockpit 
illumination  to  be  consistent  with  day,  dusk,  or  night  condi¬ 
tions.  The  image  is  projected  from  a  color  television  pro¬ 
jector  ...ounted  on  top  of  the  simulator  fuselage  through  a 
mirror  system  onto  a  back  projection  screen.  The  pilot  views 
the  visual  scene  by  means  of  a  large  collimating  mirror  posi¬ 
tioned  ahead  of  the  cockpit. 
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2 .  Computer  Generated  Imagery  (CGI) 

The  Computer  Generated  Imagery  system  consists  of  a 
numerically  stored  environment  model,  a  computation  system 
for  generation  of  display  video  from  the  numerically  stored 
environment,  and  display  electronics  for  driving  the  cathode 
ray  tube  (CRT)  display  system  [Ref.  12].  This  system  is 
presently  used  by  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  on  their  Advanced  Simulator  for  Pilot  Training  (ASPT) . 

Maneuverability  by  the  pilot  through  the  control 
system  of  the  simulated  aircraft  is  unlimited.  The  scenery 
changes  are  generated  in  response  to  the  pilot ' s  viewpoint 
position  and  attitude  in  real  time  with  eneration  of  a  com¬ 
plete  new  visual  scene  displayed  each  1/30  of  a  second  [Ref. 

13]  . 

The  physical  environment,  which  consists  of  a  flat 
surface  representing  the  surface  of  the  earth  and  three- 
dimensional  objects  representing  solid  figures,  is  numerically 
described  in  three-dimensional  vector  space. 

When  the  system  is  in  the  on-line  operational  mode, 
it  accomplishes  its  real-time  scene  generation  task  in  a  serial 
manner.  Data  that  is  necessary  for  scene  computation  is  re¬ 
quested  from  the  computer  subsystem  each  1/30  of  a  second  and 
the  corresponding  scene  is  completely  displayed  within  1/10 
of  a  second  of  receiving  new  scene  data  [Ref.  14] . 

Three  time  phases  are  used  for  processing  the  display 
data.  The  phases  are  referred  to  as  Frames  I,  II,  and  III. 
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While  Frame  III  is  producing  the  video  signal  that  is  being 
observed  by  the  pilot,  Frame  II  is  preparing  information  for 
the  picture  to  be  used  in  the  next  television  frame  period, 
and  Frame  I  is  working  on  the  most  recently  requested  data 
which  is  driven  by  the  pilot's  viewpoint  position  and 
attitude  [Ref.  15] . 

Visual  scene  development  using  a  CGI  system  has 
certain  primary  capabilities  including  exact  perspectiveness, 
real-time  display  of  moving  objects,  quick  visual  environment 
change  or  modification,  unlimited  rate  of  maneuverability, 
and  a  large  area  of  flight  coverage  [Ref.  16] .  The  imagery, 
as  viewed  by  the  pilot,  consists  of  surface  patterns  or 
objects  formed  by  planes  of  different  brightness  levels 
bounded  by  straight  lines  or  "edges."  A  system  with  a  finite 
edge  generation  capability  results  in  a  stylistic  presentation 
rather  than  according  to  nature.  As  a  rule,  the  degree  of 
stylization  is  inversely  proportional  to  the  edge  generation 
capability  of  the  system  since  scenes  in  the  real  world  are 
not  constrained  to  representation  by  straight  lines  or  edges 
[Ref.  17].  To  improve  image  quality,  two  techniques,  edge 
smoothing  and  continuous  shading  of  surfaces,  are  employed. 

The  edge  smoothing  feature  provides  a  gradual  transition 
across  an  edge  while  the  continuous  shading  of  surfaces  capa¬ 
bility  permits  the  generation  of  imagery  representing  curved 
surfaces  [Ref.  18].  The  present  system  has  the  capability 
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of  generating  2500  edges  with  2000  being  displayed  at  any  one 
time  [Ref.  19]. 

Using  the  serial  method  of  visual  scene  generation, 
the  system,  up  to  this  point,  has  been  storing,  retrieving, 
and  transforming  edge  format  information.  The  next  step  in 
the  process  is  to  convert  edge  format  into  digital  scan  line 
format  allowing  the  brightness  level  of  each  part  of  the  scan 
line  to  be  in  digital  form.  This  scan  line  information  is 
then  converted  into  a  video  signal  by  a  high-speed  digital- 
to-analog  converter.  This  signal  is  then  distributed  to  the 
fourteen  display  planes  for  viewing  by  the  pilot,  all  in  less 
than  100  milliseconds  [Ref.  20].  The  system,  as  it  presently 
exists  for  the  Advanced  Simulator  for  Pilot  Training  (ASPT) , 
has  two  cockpits,  each  with  seven  video  display  planes.  The 
planes  are  juxtapositioned  about  one  viewing  point  with  over¬ 
lapping  visual  information.  The  system  has  the  capability  of 
providing  a  minimum  of  7.2  degrees  overlap  of  visual  informa¬ 
tion  on  the  borders  of  the  seven  display  planes.  When  one 
edge  is  displayed  in  two  or  more  adjacent  channels,  the  system 
will  generat  -±  the  signal  such  that  this  edge  has  less  than  one 
degree  discontinuity  across  the  display  joint  [Ref.  21] . 

Difficult  maneuvers  to  visually  simulate,  which  in¬ 
clude  overhead  pattern  and  landing,  formation  flight,  and 
aerobatics,  can  be  accomplished  using  CGI.  For  example,  the 
pilot  can  fly  over  the  runway  at  1000  feet,  then  make  a  180 
degree  level  turn  to  the  downwind  and  fly  the  appropriate 
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landing  pattern  to  touchdown.  Throughout  this  maneuver, 
the  pilot  uses  computer  generated  visual  cues  that  are  being 
projected  on  the  seven  display  planes  to  determine  attitude 
and  position  with  respect  to  the  touchdown  point.  Precise 
aerobatic  maneuvers  can  be  accomplished  because  the  pilot 
is  able  to  look  directly  overhead  and  out  both  sides  of  the 
cockpit  for  aircraft  control  and  performance  assessment. 

During  formation  flying,  the  pilot  uses  visual  cues  from  the 
leau  aircraft  to  maintain  a  satisfactory  wing  position. 

Reliability  of  the  CGI  system  has  a  design  goal  in 
terms  of  mean-time-between-failures  of  150  hours  [Ref.  22]. 

The  system  is  also  designed  to  have  a  minimum  accumulated 
operating  life  of  30,000  hours  and  a  maintainability  require¬ 
ment  that  maintenance  man-hours,  comprised  of  corrective  and 
preventive  maintenance,  calibrations,  and  servicing,  will  not 
exceed  20  percent  of  the  accumulated  operating  time  [Ref.  23]. 

There  are  limitations  to  CGI  predominantly  in  the 

area  of  image  content  and  detail;  however,  with  the  advent 

of  present  technology,  the  system  provides  a  more  complete 

visual  simulation  in  terms  of  perspective,  field  of  view,  and 

unprogrammed  flight  conditions  [Ref.  24] .  In  a  paper  presented 

to  the  National  Aerospace  meeting,  Mr.  G.  V.  McCulloch,  United 

Airlines  Flight  Training  Center,  stated: 

United  Airlines  believes  that  computer  generated  imaging 
is  the  visual  simulation  technique  offering  the  greatest 
promise  of  making  total  airplane  simulation  possible. 

[Ref.  25]. 
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3 .  Computer  Animated  Photographic  Terrain  View  (CAP TV) 

A  third  method  by  which  visual  scenes  can  be  produced 
for  flight  simulation  is  Computer  Animated  Photographic  Ter¬ 
rain  View  (CAP TV) .  This  system  is  under  development  and  being 
considered  for  the  Navy's  new  jet  aircraft  training  program 
(VTX(TS)).  The  process  originates  by  having  an  aircraft  with 
a  camera  mounted  underneath  the  fuselage  fly  over  a  "gaming 
area"  such  as  an  aircraft  carrier,  runway,  or  target  at  dif¬ 
ferent  altitudes,  recording  the  scenes  on  motion  picture  film. 
The  camera  uses  seven  lens  systems  simultaneously  capturing 
scenes  of  360  degrees  in  azimuth  and  up  to  10  degrees  above 
the  horizon.  The  photographs  are  processed  through  a  flying 
spot  scanner,  color  encoded,  and  recorded  on  a  video  disc. 

The  video  disc  units  have  the  capability  of  recording  400  high 
fidelity  color  photographs  per  disc  surface. 

When  the  visual  scenes  are  to  be  used  for  flight 
simulations,  the  discs  are  read  optically  by  laser,  that  is, 
an  electronic  digital  picture  processor  (DPP)  accesses  photo¬ 
graphs  from  the  disc  and  interpolates  for  infinite  eyepoints. 
The  DPP  has  a  large  digital  memory  which  holds  the  last  read 
scene  of  interest  and  then  ass^bles  the  scene  as  it  is 
accessed  from  the  disc. 

Since  the  camera  mounted  on  the  aircraft  only  covers 
the  earth  and  10  <\ agrees  above  the  horizon,  computer  generated 
imagery  provides  the  sky  simulation  and  can  also  be  used  for 
insetting  such  special  effects  as  moving  targets. 
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After  the  video  special  effects  have  been  inserted, 
the  image  can  be  displayed  using  either  a  dome  projection  or 
cathode  ray  tubes.  Because  the  visual  scenes  are  actual 
photographs  cf  the  terrain  and  objects  of  interest,  CAPTV  can 
provide  highly-detailed  information,  accurate  depth  perception, 
and  real-world  scenery  texture  within  a  large  field  of  view 
[Ref.  26] . 

C.  A  DESCRIPTION  OF  MILITARY  FLIGHT  SIMULATORS 

1.  A-7E  Night  Carrier  Landing  Trainer,  Device  2F-103 

The  Night  Carrier  Landing  Trainer  (NCLT)  is  utilized 
to  simulate  an  A-7E  aircraft  flying  a  night  aircraft  carrier 
approach  and  landing.  The  system  is  composed  of  a  simulated 
A '7E  cockpit,  a  visual  display  system,  a  3  degree- of- freedom 
motion  system,  instructor  console,  digital  computer,  and 
related  hydraulic  and  electrical  supplies  [Ref.  27].  The 
visual  scene  is  displayed  on  a  single  cathode  ray  tube  which 
is  positioned  in  front  of  the  cockpit,  providing  the  pilot 
with  a  40  degree  horizontal  and  30  degree  vertical  colored 
picture  of  the  deck  lighting  and  visual  landing  aids  such  as 
the  Fresnel  Lens  Optical  Landing  System  (FLOLS) ,  which  is 
properly  positioned  in  relation  to  the  deck  edge,  the  runway 
lights,  and  vertical  drop  lights,  all  of  which  are  used  to 
form  a  basic  "night”  landing  picture.  The  computer  generates 
a  two  dimensional  perspective  view  of  the  carrier  lights  and 
updates  the  spatial  placement  as  a  function  of  closing  range 
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to  touchdown  and  high/low  or  left/right  approach  positions 
dictated  by  real-time  pilot  control  inputs  [Ref.  2 R ] . 

Through  interface  with  the  instructor  console,  the 
Landing  Signal  Officer  (LSO)  can  monitor  the  pilot’s  perfor¬ 
mance.  The  LSO  can  also  vary  the  degree  of  difficulty  of  each 
approach  by  selecting  different  environmental  characteristics 
such  as  a  rough  sea  state  or  high  wind  conditions  across  the 
aircraft  carrier  deck.  Fourteen  aircraft  emergencies  can 
also  be  programmed  during  any  approach  [Ref.  29]. 

In  order  to  improve  upon  debriefing  procedures,  the 
LSO  can  obtain  records  from  an  X-Y  plotter  whioh  shows  altitude 
and  lateral  error  deviations  from  the  desired  glideslope  and 
line  up.  In  addition  to  this  recording  capability,  the  system 
can  replay  the  last  minute  of  final  approach  and  also  "freeze" 
the  display  so  that  the  LSO  can  provide  instructional  advice 
at  the  point  in  time  where  the  pilot  commits  the  error.  In 
addition,  information  such  as  angle-of~attack  (AOA)  control, 
power  control,  pitch  control,  and  tailhook-to-ramp  clearance 
can  also  be  displayed  to  both  the  pilot  and  instructor.  If 
desired,  a  hard  copy  of  these  results  can  be  obtained  from  the 
printer. 

The  NCLT  provides  the  pilot  with  complete  freedom  to 
fly  the  aircraft  and  realistic  aircraft  sounds  throughout  the 
flight.  Carrier  arrestment  is  simulated  by  stopping  the  CRT 
display.  The  device  also  has  the  capability  of  simulating 
landing  and  missing  the  arresting  wires  with  the  aircraft 
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tailhook,  touch- and-goe s ,  ramp  strikes,  and  allows  the  pilot 
to  re-enter  the  final  approach  pattern  after  each  missed 
landing  [Ref.  30] . 

2.  A-7E  Weapons  System  Trainer,  Device  2F-111 

This  system  includes  an  A-7E  cockpit  mounted  on  a  six 
degree-of-freedom  motion  platform,  digital  computers,  a  digital 
radar  landmass  system,  interface  equipment,  and  instructor's 
console.  The  trainer  features  flight  validated  systems  to 
include  departure  from  flight,  aircraft  spin  and  recovery 
characteristics,  utilization  of  CRT  displays  for  instructor 
information,  integration  of  the  navigational  computer  with 
other  systems,  and  performance  measurement  capability  [Ref. 

31]  . 

Simulation  of  the  aircraft  is  very  precise  with  the 
system  including  all  modes  of  engine  and  fuel  system  opera¬ 
tion,  hydraulic  and  electrical  systems,  flying  qualities,  and 
performance  characteristics. 

For  tactical  instruction,  the  system  includes  simula¬ 
tion  of  weapon  delivery,  loading,  arming,  safeing,  and  ordnance 
release.  The  instructor  has  the  capability  of  simulating  any 
weapons  loading  configuration.  In  addition,  the  trainer  can 
simulate  signals  that  are  emitted  when  enemy  weapons  systems 
have  a  radar-lock-on  such  as  a  surface-to-air  missile  (SAM) 
allowing  the  pilot  to  practice  using  electronic  countermeasure 
systems  and  also  to  perform  appropriate  evasive  maneuvers. 
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The  system  also  has  a  "freeze"  capability  allowing 
the  instructor  to  completely  stop  all  systems  for  debriefing 
and  analysis  and  then  move  to  another  geographical  location 
to  continue  the  training  situation.  There  is  a  data  base  map 
within  the  system  which  depicts  a  1250  x  1250  nautical  mile 
area  of  either  the  western  or  eastern  United  States  [Ref.  32]. 

The  instructor  has  the  ability  to  enter  eighty  mal¬ 
functions  into  the  system/  increasing  the  level  of  difficulty 
for  the  pilot-under-training.  Four  CRT  displays  allow  the 
instructor  to  monitor  the  status  of  the  training  mission  and 
the  proficiency  of  the  pilot  in  accomplishing  the  mission. 
Included  within  the  displays  are  a  repeat  of  cockpit  switch 
positions,  instrument  panel  indications,  the  capability  to 
monitor  sequential  actions  by  the  pilot  for  a  given  procedure, 
and  information  relating  to  the  position  and  orientation  of 
the  aircraft  [Ref.  33] . 

3 .  Advanced  Simulator  for  Pilot  Training  (ASPT) 

The  Advanced  Simulator  for  Pilot  Training  (ASPT)  is 
used  primarily  in  the  research  area  with  formally  stated  ob¬ 
jectives  of  the  program  being:  (1)  to  enhance  pilot  training 
within  the  Air  Force  through  the  application  of  recent  tech¬ 
nological  advances  in  simulation;  (2)  to  demonstrate  the  max¬ 
imum  effective  utilization  of  simulators  in  Air  Force  pilot 
training;  and  (3)  to  define  the  future  generation  of  ground 
training  equipment  and  simulators  [Ref.  34]. 
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The  system  is  comprised  of  three  major  components: 
the  basic  simulators,  the  visual  displays,  and  the  computer 
image  generator  [Ref.  35] .  The  visual  display  and  computer 
image  generating  systems  for  the  ASPT  have  been  described  in 
a  preceding  section,  therefore,  the  basic  simulator  will  be 
described  at  this  point. 

The  ASPT  is  presently  configured  with  the  F-16  and 
the  A- 10  cockpits  enclosed  within  the  sever-channel  visual 
display  systems.  The  realistic  appearances  of  the  cockpits 
are  provided  by  the  utilization  of  actual  aircraft  parts,  and 
the  cockpits  include  faithful  reproductions  of  in-cockpit 
sights,  sounds,  and  aerodynamic  control  feel  to  the  maximum 
extent  allowable  within  the  state-of-the-art  in  simulation. 

The  rotion  system  provides  the  on-set  acceleration 
cues  along  and  about  the  three  aircraft  axes.  The  motion 
platform  is  supported  by  six  active  hydraulic  actuators  with 
six  passive  safety  actuators  providing  complete  mechanical 
redundancy  in  case  of  system  failure.  The  sixty- inch  stroke 
system  is  essentially  a  hydraulic  position  servo  that  is 
driven  by  commanded  actuator  lengths  computed  by  the  motion 
system  mathematical  model  [Ref.  36]. 

The  ASPT  also  has  provisions  for  introducing  several 
levels  of  difficulty  and  complexity  within  any  given  task. 
These  variables  are  accomplished  by  restricting  any  combin¬ 
ation  of  the  six  degrees-of-f reedom  motion  system,  varying 
aircraft  response  to  control  movements,  inserting  system 
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malfunctions,  or  introducing  environmental  factors  such  as 
temperature  changes,  turbulence,  and  wind  velocity. 

The  ASPT  has  the  capability  to  " freeze"  the  simulated 
visual  scene,  reinitialize  to  a  point  in  space,  automatic 
demonstration,  provide  the  stude.nt  with  knowledge  of  the 
results,  and  playback  of  a  particular  maneuver. 

Unlike  other  flight  simulation  systems  described  in 
this  section,  the  ASPT,  through  Computer  Generated  Imagery, 
can  change  the  training  environment  almost  instantaneously. 

This  author  had  the  opportunity  to  “fly"  the  A-10  ASPT  during 
a  recent  visit  to  the  Human  Resources  Laboratory,  Williams  AFB, 
Arizona.  During  the  sixty  minute  training  period,  the  author 
flew  in  four  different  environments  to  include  aircraft  carrier 
approaches  and  landings,  formation  flying  with  a  simulated 
A-10  aircraft,  air-to-air  refueling  using  a  simulated  KC-135 
tanker,  and  air-to-ground  ordnance  delivery  in  a  hostile  envi¬ 
ronment..  While  in  the  last  three  environments,  the  computer 
demonstrated  a  “perfect"  maneuver  involving  all  instrument 
readings  and  visual  scenes  of  the  total  simulator  system. 

When  first  shown  the  CGI  environment,  the  author  was  concerned 
with  the  lack  of  realism.  However,  after  becoming  involved  in 
performing  the  required  tasks  such  as  flying  the  approach  to 
the  aircraft  carrier,  the  realistic  aspect  of  the  visual  scene 
became  secondary  in  importance  while  the  correct  performance 
of  the  maneuver  was  of  primary  interest. 
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D.  SUMMARY 


Part  A  of  this  section  outlined  the  historical  develop¬ 
ment  of  flight  simulation  from  the  earliest  devices  around 
1910  using  actual  aircraft  to  the  modern  flight  simulator 
which  uses  sophisticated  computing  techniques  to  animate  full 
scale  representations  of  the  operational  environment  experi¬ 
enced  by  the  pilot  during  flight.  Part  B  described  different 
methods  being  utilized  today  to  provide  visual  flight  simula¬ 
tion.  The  methods  included  model  boards#  computer  generated 
imagery  (CGI) ,  and  computer  animated  photographic  terrain  view 
(CAPTV) .  Part  C  presented  a  description  of  three  military 
flight  simulators  including  the  A-7E  Night  Carrier  Landing 
Trainer#  the  A-7E  Weapons  System  Trainer,  and  the  Advanced 
Simulator  for  Pilot  Training  being  used  by  the  Air  Force  for 
visual  flight  simulation  research. 


III.  FLIGHT  SIMULATOR  UTILIZATION  IN  THE 
COMMERCIAL  AVIATION  INDUSTRY 

A.  GENERAL 

The  use  of  flight  simulators  to  provide  initial  and 
recurrent  training  for  flight  crews  of  commercial  airline 
companies  has  increased  rapidly  since  World  War  II.  The 
airline  companies  saw  flight  simulator  utilization  as  an 
opportunity  to  reduce  the  costs  associated  with  training  their 
Captains,  First  Officers,  and  Flight  Engineers.  Improvements 
in  flight  simulation  hardware,  supported  by  more  complete 
performance  data  from  aircraft  flight  tests,  and  fostered  by 
a  more  permissive  Federal  Aviation  Administration  (FAA)  regu¬ 
latory  environment,  have  moved  the  air  transport  industry 
closer  to  an  ultimate  goal  of  total  flight  training  through 
simulation  [Ref.  37]. 

The  impetus  to  use  flight  simulators  has  been  provided 
by  three  primary  considerations,  namely,  safety,  simulator 
quality,  and  energy  conservation.  Within  the  safety  regime, 
pilots  are  able  to  practice  unique  maneuvers  such  as  power 
loss  after  takeoff  commital  speed,  and  limited  power  on  final 
approach  to  landing.  Simulator  training  is  especially  imper¬ 
ative  for  Flight  Engineers,  because  it  permits  shut-down  of 
electrical  or  hydraulic  systems  without  exposing  the  aircraft 
to  undue  hazards.  An  additional  effective  utilization  of 
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simulators  to  improve  safety  conditions  would  be  using  the 
devices  to  reconstruct  events  as  they  occurred  during  flight. 

To  illustrate,  on  May  25,  1979,  an  American  Airlines  DC-10 
crashed  after  losing  the  left  engine  on  takeoff  from  Chicago's 
O' Hare  Airport.  Technicians  took  data  from  the  flight  recorder 
of  the  crashed  aircraft  and  translated  the  information  into 
a  computer  to  be  used  with  a  DC-10  simulator  to  try  to  deter¬ 
mine  why  the  pilot  could  not  control  the  left-  roll,  and  if  any 
combination  of  aerodynamic  control  inputs  or  throttle  positions 
of  the  remaining  operable  engines  could  have  prevented  the 
accident  [Ref.  38]. 

New  simulation  equipment  that  is  being  procured  by  the 
major  airline  carriers  is  inherently  capable  of  fulfilling  PAA 
advanced  simulation  requirements.  State-of-the-art  simulators 
use  computer  generated  imagery  and  six  degrees-of-freedom 
motion  systems.  There  are  normally  four  visual  screens  posi¬ 
tioned  in  front  of  and  on  the  left  side  of  the  Captain  and  in 
front  of  and  on  the  right  side  of  the  First  Officer.  These 
side  screens  allow  the  pilots  to  practice  circling  approaches 
instead  of  being  restricted  to  straight-in  approaches  to  land¬ 
ing.  FAA  requirements  will  be  explained  in  detail  in  a  subse¬ 
quent  part  of  this  chapter. 

During  the  early  stages  of  flight  simulator  development 
aircraft  costs  incurred  to  practice  procedures  and  build 
practical  experience  were  less  expensive  than  investments  in 
peripheral  training  equipment.  As  a  result  of  escalating  fuel 
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costs  and  increased  technology ,,  this  trend  has  reversed  itself. 
The  Boeing  Corporation  determined  that  visual  simulators  have 
trimmed  12  of  the  18  flight  hours  formerly  scheduled  for  each 
pilot  transitioning  to  one  of  their  jet  aircraft.  As  an 
example,  the  cost  associated  with  transition  training  in  a 
Boeing  727  is  $1400  per  flight  hour  in  the  aircraft  versus 
$280  per  flight  hour  in  the  simulator  [Ref.  39].  Without  the 
simulator  the  training  cost  would  be  $1400  per  hour  x  18  hours 
*  $25,200.  With  the  simulator/aircraft  combination  the  train¬ 
ing  cost  would  be  $1400  per  hour  x  6  hours  plus  $280  per  hour 
x  12  hours  «  $11,760.  This  results  in  a  savings  of  $13,440 
for  each  pilot  trained.  Data  obtained  from  a  major  airline 
company  showed  that  their  short-haul  aircraft  (500  nautical 
miles  between  landings)  cost  $1400  per  flight  hour  while  the 
simulator  cost  was  $250  per  hour,  and  their  long-haul  aircraft 
(in  excess  of  1100  nautical  miles  between  landings)  cost  $5600 
per  flight  hour  while  the  simulator  was  $300  per  hour.  For 
qualification  as  Captain  in  the  short-haul  aircraft,  a  pilot 
would  require  approximately  12  hours  in  the  aircraft  only 
($16,800)  or  16  hours  in  the  simulator  plus  three  hours  in 
the  aircraft  ($8,200).  In  the  long-haul  aircraft,  qualifi¬ 
cations  would  require  approximately  12  hours  in  the  aircraft 
only  ($67,200),  or  15  hours  in  the  simulator  plus  two  hours 
in  the  aircraft  ($15,700) •  Based  on  822  crew  qualifications 
per  year  in  the  short-haul  aircraft  and  411  qualifications 
per  year  in  the  long-haul  aircraft,  this  major  airline 
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company  recognized  potential  annual  cost-savings  associated 
with  training  of  $7,069,200  and  $21,166,500  respectively  in 
1980  [Ref.  40]. 

Flight  simulation  technology  has  progressed  from  a  simple 
fixed  base  of  a  single  place  cockpit  to  a  device  capable  of 
immersing  a  pilot  and  his  crew  in  a  very  realistic  and  typical 
flight  situation.  Detailed  environmental  situations  such  as 
communication  with  the  ground  crew  during  "pushback,"  reaction 
of  the  aircraft  to  wind  shear,  any  level  of  aircraft  system 
malfunction,  ground  effect,  and  braking  deceleration  on  land¬ 
ing  rollout  can  be  realistically  simulated  with  state-of-the- 
art  technology  [Ref.  41].  American  Airlines  has  conducted  at 
least  eighty  requalifications  of  pilots  who  had  been  away  from 
line  duty  for  periods  of  ninety  days  to  two  years  with  complete 
retraining  in  a  simulator  under  an  FAA  exemption  and  with  no 
non-revenue  flight  time.  American  also  conducted  upgrade 
studies  taking  First  Officers  who  were  current  within  the  past 
two  years  as  pilots  of  airplanes  of  the  same  type  and  upgrad¬ 
ing  to  Captain  with  no  aircraft  non-revenue  time  [Ref.  42]. 

Commercial  airlines  use  of  flight  simulators  is  a  multi¬ 
million  dollar  business;  therefore,  a  tremendous  effort  is 
put  forth  in  organizing  and  operating  the  training  centers 
which  utilize  the  simulators.  The  next  part  of  Section  III 
will  describe  such  a  training  center. 
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B.  THE  FLIGHT  TRAINING  CENTER  CONCEPT 

Extensive  research  by  this  author  has  shown  that  major 
airlines  have  tended  to  centralize  their  training  facilities 
in  order  to  take  advantage  of  economies  of  scale  and  to  im¬ 
prove  standardized  operating  procedures.  In  1966,  as  a  result 
of  a  study  completed  by  a  private  consulting  firm,  American 
Airlines  centralized  their  training  program  in  the  Dallas-Fort 
Worth  area.  This  change  had  been  precipitated  by  the  fact  that 
two  fatal  training  accidents  had  recently  occurred  with  a  pri¬ 
mary  cause  being  a  difficulty  in  maintaining  standardization, 
since  at  that  time  American  had  training  facilities  in  New 
York,  Chicago,  Los  Angeles,  and  Dallas-Fort  Worth.  The  major 
complex  was  completed  in  1970  and  trains  approximately  4000 
flight  crewmen  annually  through  any  of  75  different  types  of 
flight  and  ground  school  courses  covering  initial,  refresher, 
upgrade,  and  transition  training  [Ref.  43]. 

Figure  III-l  depicts  an  organization  chart  for  a  major 
airline  flight  training  center.  It  is  readily  apparent  that 
the  training  center  is  a  very  large  organization  requiring  a 
tremendous  amount  of  coordination  between  major  departments 
to  ensure  that  operations  are  both  effective  and  efficient. 
The  Standards  and  Procedures  Department  performs  all  the 
necessary  flight  checks,  both  within  the  simulator  and  the 
aircraft,  in  order  to  ensure  that  the  flight  crews  perform 
the  procedures  correctly.  Flight  Simulator  Services  attest 
to  the  fidelity  of  the  flight  simulators,  perform  necessary 
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system  changes,  and  ensure  all  systems  are  operating  within 
FAA  specifications.  The  Director  of  Flight  Operations  coor¬ 
dinates  the  input  and  training  of  all  the  crews  ensuring  that 
an  individual's  program  is  progressing  according  to  the  appro¬ 
priate  syllabus.  The  Manager  of  Flight  Training  Programs 
maintains  the  present  training  syllabus  and  develops  new  tech¬ 
niques  for  improving  classroom  and  other  media  instruction. 
Finally,  the  Manager  of  Avionics  is  responsible  for  the  main¬ 
tenance  effort  associated  with  maintaining  the  simulators  and 
other  training  devices. 

Two  major  contributions  to  the  success  of  the  flight 
training  center  concept  have  been  the  instructor  staff  and 
up-to-date  training  systems  and  techniques  for  using  these 
systems.  The  instructors  are  professionals  with  extensive 
backgrounds  in  classroom  teaching  in  addition  to  considerable 
flight  experience,  averaging  15,000  to  20,000  flight  hours. 
Training  techniques,  such  as  the  concept  of  individualized 
training,  has  received  much  emphasis.  This  type  of  training 
permits  each  student  to  proceed  at  his  own  best  learning  rate. 
Subjects,  such  as  aircraft  systems,  are  divided  into  study 
units.  Each  study  unit  is  intended  to  teach  and  test,  with 
testing  being  accomplished  through  the  timely  display  of  mul¬ 
tiple  choice  questions.  If  the  student  answers  the  questions 
correctly,  the  study  unit  automatically  continues.  Post¬ 
training  performance  has  shown  an  exceptionally  high  level  of 
retention.  The  training  technique  also  allows  the  student  to 
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progress  from  the  aircraft  system  study  units  through  a 
systems  trainer,  the  Cockpit  Procedures  Trainer  (CPT) ,  and 
finally  to  the  Pull-Mission  Simulator.  This  procedure  used 
by  the  commercial  aircraft  industry  ensures  that  operations 
of  all  the  training  devices  are  cost-effective. 

Normally,  aircrews  return  to  the  flight  training  center 
every  six  months.  One  of  the  sessions  includes  an  extensive 
briefing  of  commercial  aircraft  accidents  or  incidents,  an 
operational  review,  and  the  yearly  proficiency  check  required 
by  FAA.  The  second  session  includes  aircraft  evacuation 
training,  aircraft  systems  review,  and  line-oriented  flight 
training  (LOFT) .  LOFT  creates  line  conditions  during  a  sim¬ 
ulator  flight  and  allows  the  crew  to  face  typical  training 
problems.  Any  malfunctions,  diversions,  or  other  circum¬ 
stances,  such  as  a  passenger  experiencing  a  heart  attack,  may 
be  introduced  during  the  course  of  the  four  hour  flight.  The 
crew  undergoing  training  must  deal  with  the  problems  themselves 
which  demonstrates  the  ability  of  the  Captain  to  effectively 
use  the  human  resources  available.  The  LOFT  instructor  may 
not  add  any  information  or  advice  and  may  not  interrupt  the 
exercise  to  critique  any  actions  until  the  simulation  has  been 
completed  [Ref.  44] . 

C.  FAA  RULINGS  CONCERNING  VISUAL  FLIGHT  SIMULATION 

The  FAA  has  proposed  new  rules  dealing  with  progress 
toward  100  percent  simulator  programs.  The  program  is  divided 


into  basically  three  phases.  Phase  1  requires  that  the  simu¬ 
lators  be  upgraded  to  the  best  that  presently  exists,  with 
the  capability  of  successful  training  in  takeoff  and  landing 
maneuvers  and  ground  handling  of  the  aircraft.  A  Phase  2 
simulator  would  allow  the  airlines  the  ability  to  upgrade  to 
Captain  a  pilot  currently  qualified  as  a  First  Officer  on  the 
same  type  of  aircraft.  It  would  also  allow  lateral  transition 
of  a  Captain  or  First  Officer  to  a  different  aircraft.  Follow¬ 
ing  completion  of  transition  training,  Captains  fly  25  hours 
and  First  Officers  fly  15  hours  under  the  supervision  of  a 
check  pilot  on  regular  revenue  generating  flights.  Phase  3 
is  defined  as  total  simulation  training  with  corresponding 
requirements  for  improved  aircraft  data  and  better  visual 
systems  available  to  all  crew  members,  including  side  window 
visuals.  In  1979,  Pan  American  was  granted  an  exemption  to 
perform  a  study  to  determine  the  feasibility  of  total  simu¬ 
lator  training.  A  group  of  118  pilots  was  divided  into  a 
control  group  and  experimental  group  with  the  experimental 
group  completing  the  transition  training  without  any  aircraft 
time.  At  the  end  of  the  training,  both  groups  were  examined 
thoroughly  by  the  FAA  and  Pan  American  check  pilots  with  no 
discernible  difference  found  in  their  performance.  Excellent 
training  results  such  as  reported  by  Pan  American  point  to 
.implementation  of  Phase  3  training  as  a  realistic  objective 
by  1982-83  [Ref,  45] . 
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In  a  recent,  interview  with  Mr.  William  Traub,  Director 
of  Flight  Operations,  United  Airlines,  the  cost  savings  asso¬ 
ciated  with  upgrading  the  simulators  to  Phase  2  was*  outlined 
as  follows: 


B-727  AIRPLANE  TRAINING  AMD  CHECKING 


1979  (Year  of  normal  expansion  and  training) 

1.  Total  Aircraft  Time  -  694  hours 

2.  Total  Cost  to  Fly  Above  Hours  -  $1,058,350 


3.  Fuel  Burned  - 


5,968,400  lbs. 


1980  (Year  of  limited  training) 

1.  Total  Aircraft  Time  - 

2.  Total  Cost  to  Fly  Above  Hours  - 

3.  Fuel  Burned  - 


434  hours 
$661,850 
3,732,400  lbs. 


This  requirement  to  use  the  actual  aircraft  would  have  been 
alleviated  with  Phase  2  simulators.  Appendix  A  of  this  thesis 
describes  the  simulator  and  visual  system  requirements  for  the 
three  phases,  as  presented  in  Appendix  H  to  Part  121  of  the 
Federal  Aviation  Regulations. 


D.  A  DESCRIPTION  OF  FLIGHT  SIMULATION  SYSTEMS 

The  approach  toward  Phase  3  simulation  has  been  made  pos¬ 
sible  by  advances  in  several  different  fields  including  com¬ 
puter  programming ,  hydrostatic  motion  systems,  daylight 
computer  generated  image  visual  displays,  available  aircraft 
data,  instructor  integration,  and  maintenance  [Ref.  46] . 

Accurate  aircraft  data  have  allowed  the  simulator  to 
duplicate  the  aircraft  more  accurately,  particularly  in 
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ground  effect  and  ground  handling.  Aerodynamic  character¬ 
istics  of  an  aircraft  change  when  within  approximately  300 
feet  of  the  ground  and  has  been  difficult  to  simulate  until 
the  present  state-of-the-art.  Better  visual  and  motion 
systems  are  now  simulating  grour.  handling  more  accurately, 
and  several  airlines  have  opted  for  four  or  even  six  window 
displays  to  provide  pilots  with  an  all-around  cockpit  view. 
Ground  handling  may  also  include  the  effect  of  ice  and  snow 
[Ref.  47]. 

Improvements  such  as  hydrostatic  technology  have  been 
used  to  reduae  friction  in  motion  and  control  feel  systems. 
Prior  to  this  new  technology,  the  pilot  would  notice  a  dis¬ 
concerting  "bump"  especially  during  the  return  to  the  neutral 
position  by  the  jack  after  the  system  had  inputted  a  particu¬ 
larly  large  acceleration.  Hydrostatic  jacks  reduce  friction 
to  about  one-tenth  of  its  previous  value  by  eliminating  direct 
contact  between  piston  and  cylinder  wall.  High  pressure 
hydraulic  fluid  is  leaked  between  the  piston  and  wall  and  the 
pressure  centralizes  the  piston  because  it  acts  equally  in 
all  directions  [Ref.  48] . 

Control  feel  errors  he  /e  been  reduced  to  about  one-half 
their  previous  value  as  a  result  of  the  simulator  companies 
adopting  electronic  feedback  controls.  These  components  allow 
technicians  to  individually  alter  a  particular  parameter  with¬ 
out  introducing  undesirable  effects  in  other  areas.  This  has 


increased  significantly  the  time  the  simulator  has  been  made 
available  to  the  user  airline  [Ref.  49], 

Major  airlines  are  emphasizing  instructor  integration 
into  the  cockpit  while  the  student  is  undergoing  training. 

This  allows  the  instructor  to  devote  more  time  to  observing 
and  instructing.  Pre-programmed  aircraft  system  faults,  hand¬ 
held  controllers,  and  full  color  CRT  displays  are  training 
aids  made  available  in  the  cockpit.  An  instructor  could,  for 
example,  select  a  pre-programmed  engine  failure  to  occ  ia  at  a 
particular  altitude  after  takeoff.  He  could  observe  the 
method  by  which  the  crew  complied  with  the  emergency  proced¬ 
ures,  and  without  taking  his  eyes  off  them,  input  other  faults 
through  the  hand-held  controller.  Both  the  crew  and  instructor 
could  debrief  the  procedure  by  using  the  CRT  and  a  printed  copy 
of  the  plot  of  the  aircraft  position  could  be  made,  if  desired 
[Ref.  50]. 

Most  airline  simulators  are  presently  using  visual  systems 
with  night/dusk  CGI  capability,  using  6,000  to  10,000  light 
points  to  create  the  image;  however,  better  computers  are  being 
developed  which  will  allow  daylight  full  color  imagery  to  be 
developed.  The  FAA's  total  simulation  plan  (Phase  3)  requires 
daylight  CGI  which  typically  costs  twice  as  much  as  the  night/ 
dusk  system. 

Other  improvements  to  simulator  visual  displays  include 
using  multi-window  displays  increasing  the  crew's  field-of- 

view,  and  also  decreasing  the  gap  between  the  windows  from 
five  degrees  to  about  one  degree  [Ref.  51]. 
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Airline  companies’  utilization  of  sophisticated  flight 
simulators  must  be  carefully  scrutinized  since  the  lead  time 
is  about  two  years  and  a  typical  cost  is  about  $5  million  to 
$6  million.  Typical  cost  of  a  two-window,  night/dusk  CGI  is 
more  than  $500,000  while  a  four-window  system  would  exceed 
$1  million  (1980  dollars)  (Ref.  521.  With  costs  of  this 
magnitude,  it  is  absolutely  imperative  that  the  simulators 
be  an  effective  asset  to  flight  crew  training.  The  next  part 
of  this  Section  examines  the  training  effectiveness  of  flight 
simulators  realized  by  the  major  airline  companies. 

E.  TRAINING  EFFECTIVENESS 

The  state-of-the-art  technology  in  flight  simulation  sys¬ 
tems  combined  with  the  professional  training  concept  outlined 
in  Part  B  of  this  Section  provides  the  basic  foundation  for 
training  an  airline  pilot;  however,  the  real  test  is  to  eval¬ 
uate  whether  the  skills  learned  will  transfer  to  the  aircraft. 

Many  studies  have  been  conducted  by  major  airline  companies 
to  determine  whether  or  not  flight  simulators  can  provide  the 
necessary  training  to  economically  justify  the  costs  of  a 
visual  simulator  system,  and  more  importantly,  to  determine 
whether  critical  maneuvers  that  would  oe  unsafe  in  the  aircraft 
could  be  satisfactorily  learned  in  the  simulator.  This  part 
of  Section  III  will  review  three  such  studies. 

The  first  study  to  be  reviewed  concerns  the  requirement 
for  aircrews  to  conduct  three  takeoffs  and  landings  in  the 


47 


actual  aircraft  as  a  fundamental  requirement  to  maintain  or 
reinstate  the  pilot's  currency  in  that  particular  type  air¬ 
craft.  American  Airlines  questioned  the  effectiveness  of  this 
requirement  since  experience  had  shown  that  requalification 
flights  were  normally  made  on  days  with  excellent  weather 
conditions  and  at  unrealistic  airplane  weights.  If  the  re¬ 
qualification  flights  were  performed  in  visual  simulators , 
the  pilot  could  demonstrate  proficiency  in  both  day  and  night 
conditions  and  at  aircraft  weights  normal  to  line  operations. 
To  increase  the  completeness  of  training,  varying  conditions 
such  as  reduced  visibility  and  crosswinds  could  be  introduced 
by  using  the  simulator. 

American  Airlines  used  the  two-group  concept  in  conducting 
the  study.  The  control  group  complied  with  Federal  Aviation 
Administration  regulations  in  conducting  their  takeoff  and 
landing  requirements  while  the  study  group  used  an  approved 
digital  flight  simulator  with  a  color  visual  system.  After 
the  appropriate  training  both  groups  had  their  first  two 
landings  evaluated  by  an  approved  check  pilot.  There  were 
80  pilots  in  each  group  with  the  pilots  coming  from  all  types 
of  aircraft  including  the  Boeing  727,  707,  747,  and  McDonnell- 
Douglas  DC-10.  The  training  provided  each  pilot  was  dependent 
on  the  time  period  since  the  last  currency  rating. 

During  the  training  period  the  FAA  observed  13  line  take¬ 
offs  and  landings  for  the  control  group  and  81  line  takeoffs 
and  landings  for  the  study  group.  The  average  grades  for  the 
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line  takeoff  and  landings  for  the  control  group  were  3.33 
and  3. 40  while  the  study  group  had  average  grades  of  3.49 
and  3.60  respectively.  These  scores  were  based  on  a  five 
point  rating  scale.  It  should  alsio  be  noted  that  weather 
conditions  for  both  groups  during  the  evaluations  included 
day,  night,  dusk,  dawn,  and  a  variety  of  visibility  and 
crosswind  conditions  [Ref.  533 . 

Questionnaires  concerning  the  simulator  training  were 
completed  by  41  of  the  80  subjects  within  the  study  group. 
Forty-four  percent  of  the  pilots  favored  the  simulator  pro1' 
gram  without  any  reservations  and  an  additional  37  percent 
accepted  the  simulator  program  as  adequate  in  view  of  the 
economic  situation  or  with  some  added  qualifications,  one  of 
which  was  that  a  minimum  level  of  pilot  experience  he  required 
before  using  the  simulator  in  lieu  of  the  airplane.  The  most 
common  recommendation  was  a  minimum  of  100  hours  pilot  experi¬ 
ence  in  that  type  aircraft  [Ref.  54]. 

Based  on  analysis  obtained  from  this  study,  American  Air¬ 
lines  recommended  to  the  FAA  that  pilots  who  had  not  completed 
three  takeoffs  and  landings  within  90  days  be  allowed  to  re¬ 
qualify  using  a  visual  simulator  training  program.  This 
recommendation  was  accepted  and  has  been  incorporated  within 
the  regulations  as  long  as  the  simulator  system  has  been 
approved  by  the  FAA.  Requalification  in  the  aircraft  is  a 
very  costly  program,  therefore,  this  ruling  has  proven  to  be 
an  economical  improvement  for  the  commercial  airline  industry. 


49 


Practicing  maneuvers  in  the  simulator  that  would  be  un¬ 
safe  in  the  aircraft  is  a  second  positive  area  supporting 
training  effectiveness.  During  the  24  month  period  between 
July  1965  and  July  1967,  38  percent  of  jet  accidents  resulting 
in  fatalities  or  airframe  destruction  occurred  during  training. 
Of  these  accidents,  37.5  percent  occurred  when  the  pilot  was 
practicing  a  simulated  engine  failure  on  takeoff  [Ref.  55]. 

Between  September  1,  1971  and  January  12,  1972,  American 
Airlines  conducted  training  in  the  simulated  engine  failure 
maneuver  through  extensive  use  of  visual  flight  simulation. 

The  following  groups  of  pilots  were  studied: 


DC- 10 

Captains  - 

37 

DC- 10 

Co-pilots  - 

24 

B-747 

Captains  - 

8 

B-747 

Co-pilots  - 

5 

B-707 

Captains  - 

38 

B-707 

Co-pilots  - 

7 

B-727 

Captains  - 

38 

B-727 

Co-pilots  - 

18 

After  simulator  training,  the  pilots  were  checked  in  the 

aircraft  by  qualified  examiners  with  the  following  results: 

AVG.  NO.  OF 
PRACTICE  PERIODS 


PASS 

FAIL 

%  PASS 

IN  THE  SIMULATOR 

DC- 10 

Captains 

36 

1 

98 

5.2 

DC- 10 

Co-pilots 

24 

0 

100 

5.5 

B-747 

Captains 

8 

0 

100 

6.4 

B-747 

Co-pilots 

5 

0 

100 

5.2 

B-707 

Captains 

36 

2 

95 

6.3 

B-707 

Co-pilots 

7 

0 

100 

5.8 

B-727 

Captains 

38 

0 

100 

5.0 

B-727 

Co-pilots 

18 

0 

100 

4.4 

In  addition  to  the  Pass/Fail  score  given,  all  examiners 
graded  pilots  in  four  areas  including:  lateral  control, 
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heading  control,  climb  speed,  and  procedures.  There  was  a 
general  consistent  relationship  between  grades  given  in  the 
simulator  and  on  the  first  performance  in  the  aircraft.  Data 
actually  showed  that  pilot  performance  in  the  aircraft  was 
slightly  higher  than  in  the  simulator. 

As  a  result  of  the  success  of  this  study,  it  was  recom¬ 
mended  that  all  training  and  checking  for  maneuvers  involving 
power  plant  and  critical  systems  failures  be  conducted  in 
simulators  equipped  with  visual  systems  [Ref.  56] . 

Beginning  in  June  1967,  four  major  airline  carriers 
participated  in  a  study  determining  the  appropriate  methods 
for  training  pilots  in  landing  three  and  four  engine  aircraft 
with  50  percent  power  available.  Realizing  that  all  carriers 
did  not  have  the  same  types  of  training  equipment,  three  pro¬ 
grams  were  used  to  gather  data  for  the  report.  There  were 
three  prerequisites  required  for  each  program  including;  (1) 
a  high  level  of  proficiency  in  normal  approach  and  landing 
before  using  50  percent  power;  (2)  performance  characteristics 
of  50  percent  power  aircraft  must  be  completely  understood  by 
the  pilot  in  training;  and  (3)  a  high  proficiency  standard  of 
operating  and  landing  an  aircraft  with  50  percent  power  loss 
must  be  attained  [Ref.  57] . 

The  programs  included;  (1)  a  simulator  (without  visual) 
and  aircraft  combination  with  subcomponents  of  the  maneuver 
such  as  configuration  changes  being  mastered  in  the  simulator 
and  with  actual  maneuvers  in  the  aircraft  being  accomplished 
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at  altitude;  (2)  complete  training  in  a  visual  simulator  with 
satisfactory  performance  on  normal  and  one  engine  inoperative 
landing  being  required  before  a  50  percent  power  loss  landing 
was  attempted;  and  (3)  complete  training  in  the  aircraft  with 
individual  components  being  taught  and  integrated  at  altitude 
prior  to  actually  performing  the  landing. 

A  very  important  result  of  this  study  showed  that  a  high 
degree  of  success  was  achieved  with  each  of  the  programs,  em¬ 
phasizing  the  fact  that  flight  simulators  had  proven  to  be  an 
ideal  device  for  combining  knowledge  and  skill  requirements 
necessary  to  execute  the  50  percent  power  loss  on  landing 
maneuver.  With  the  effectiveness  results  of  the  three  programs 
being  the  same,  the  airline  companies  using  the  "simulator  only" 
training  method  showed  a  significant  cost  savings  as  a  result 
of  fewer  actual  aircraft  hours  being  utilized. 

F.  COST-SAVINGS  OF  FLIGHT  SIMULATORS 

This  part  of  Section  III  will  quantitatively  show  the 
cost  savings  realized  as  a  result  of  data  obtained  from  both 
American  and  Delta  Airlines.  In  the  case  of  American  Airlines 
data,  the  author  concentrated  on  the  costs  associated  with 
transitioning  a  Captain  from  one  aircraft  type  to  another  type. 

Table  III-l  provides  data  for  three  aircraft  flown  by 
American  Airlines,  In  order  to  conduct  a  direct  operating  cost 
comparison,  this  author  made  the  assumption  that,  if  a  simu¬ 
lator  was  not  available,  each  simulator  training  hour  would 
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have  been  performed  in  the  aircraft.  Table  III-2  shows  this 
direct  operating  cost  comparison. 

For  American  Ai.Jines,  the  direct  operating  costs  associ¬ 
ated  with  using  the  simulator  and  aircraft  in  the  training 
program  was  only  9.9  percent  of  the  costs  that  would  have 
been  incurred  if  only  the  aircraft  were  used.  It  must  also 
be  remembered  that  this  was  only  for  Captains  transitioning 
to  another  type  of  aircraft. 

The  Assistant  Manager  of  Flight  Training,  Delta  Airlines, 
provided  this  author  with  training  and  cost  data  for  all  line 
aircraft  flown  by  the  company.  Table  III-3  outlines  the  direct 
cost  per  hour  of  aircraft  training  versus  simulator  training 
for  Delta's  current  fleet  of  aircraft.  The  cost  of  fuel,  oil, 
and  taxes  is  based  on  a  fuel  price  of  $1.05  per  gallon,  plus 
$.0245  per  gallon  for  oil  and  taxes.  Maintenance  burden  costs 
are  computed  at  60  percent  of  the  fully  allocated  rate. 

Table  III-4  provides  cost  savings  associated  with  using 
the  simulator  systems  for  the  different  aircraft.  The  hours 
shows.  for  each  simulator  system  are  the  total  hours  used  to 
train  all  crewmembers  in  that  particular  aircraft.  Delta 
confirmed  the  assumption  made  by  this  author  that  if  the  sim¬ 
ulator  were  not  available,  those  training  hours  would  be  flown 
in  the  aircraft.  Consequently,  the  cost  per  hour  to  compute 
the  savings  that  result  is  the  net  aircraft  cost  calculated 
in  Table  III-3. 


TABLE  III- 3 


COST  COMPARISON  OF  AIRCRAFT  TRAINING 
VERSUS  SIMULATOR  TRAINING 
(Source:  Delta  Airlines) 


Cost  Per  Hour  for  Aircraft 

Fuel,  Oil,  and  Taxes 
Direct  Maintenance 
Maintenance  Durden 

Total  Aircraft  Cost 

Simulator  Cost  Per  Hour 

Maintenance 
Net  Aircraft  Cost 


DC- 9 

DC- 8 

B-727 

L-1011 

$  987 
130 
81 

$2050 

205 

143 

$1603 

110 

64 

$2547 
385 
19  J 

$1198 

$2398 

$1777 

$3125 

in 

00 

</> 

$  85 

$  85 

$  85 

$1113 

$2313 

$1692 

$3040 

TABLE  I I I- 4 
COST  SAVINGS 

(Source:  Delta  Airlines) 


Annual 

Aircraft  Simulator  Hours  Net  Aircraft  Cost  Savings 

DC-9  3,012  $1113  $  3 , 352, 356 

DC- 8  1,315  2313  3,041,595 

B-727  12,571  1632  21,270,132 

L-1011  3,300  3040  10,032,000 

Total  Savings  -  $37,696,083 


The  cost  (1980)  of  procuring  the  simulator  systems  that 
are  being  utilized  by  Delta  Airlines  is  approximately 
$41,500/000.  Since  the  savings  were  $37,696,083,  this  would 
allow  the  initial  cost  to  be  recouped  in  approximately  1.10 
years . 

G.  SUMMARY 

This  Section  has  described  the  use  of  visual  flight  simu¬ 
lators  by  major  commercial  airline  companies.  The  companies 
have  consolidated  their  initial,  transition,  and  recurrent 
training,  taking  advantage  of  standardization  and  economies 
of  scale.  The  FAA  has  worked  in  close  harmony  with  the  air¬ 
lines  so  that  the  simulator  systems,  with  their  potential  for 
training,  could  be  used  in  the  most  effective  and  efficient 
manner.  Training  effectiveness  was  successfully  measured 
using  simulator  systems  in  such  areas  as  takeoff  and  landing 
requalification  and  practicing  the  hazardous  maneuvers  of 
losing  an  engine  once  the  aircraft  reached  takeoff  commital 
speed  and  50  percent  power  loss  during  final  approach.  The 
final  part  of  Section  III  quantitatively  describes  cost 
savings  that  have  been  realized  by  two  major  air  carriers. 

There  are  basically  two  areas  which  highlight  major  dif¬ 
ferences  between  commercial  airlines  and  military  utilization 
of  visual  flight  simulators.  First,  commercial  airlines  do 
not  get  involved  in  the  initial  training  of  a  pilot.  For  the 
most  part,  airline  companies  hire  pilots  with  many  hours  of 
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military  or  private  flying.  Secondly,  the  missions  are 
entirely  different.  Simulating  air-to-ground  delivery  of 
ordnance  or  air-to-air  combat  is  an  entirely  different 
environment  than  teaching  someone  to  fly  an  approach  and 
landing  to  a  major  aerodrome  such  as  San  Prancisco  or  Denver. 
Can  visual  flight  simulation  be  effective  when  used  in  mil¬ 
itary  training?  Section  IV  outlines  training  effectiveness 
of  visual  flight  simulators  within  the  military. 
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XV.  TRAINING  EFFECTIVENESS  OF  MILITARY  FLIGHT  SIMULATORS 


A.  GENERAL 

History  indicates  that  the  single  most  decisive  factor 
in  winning  and  losing  wars  is  the  adequacy  of  training  and 
the  motivation  of  the  personnel  who  make  up  the  combat  forces. 
During  peacetime,  the  realism  of  training  is  difficult  to 
maintain  without  jeopardizing  the  safety  of  these  personnel 
and  their  valuable  equipment.  Because  of  this  problem,  the 
Department  of  Defense  has  invested  millions  of  dollars  to 
procure  simulators  which  assume  to  provide  realistic  and 
effective  training  at  reduced  costs.  As  a  result,  many  stud¬ 
ies  have  been  conducted  to  gather  and  analyze  data  in  order 
to  determine  quantitatively  and  qualitatively  that  simulators 
are  devices  that  enhance  training  effectiveness. 

Simulator  effectiveness  is  typically  assessed  through  the 
use  of  a  transfer  of  training  paradigm  [Ref.  58] .  Experimental 
and  control  groups  are  normally  evaluated  both  objectively  and 
subjectively  in  order  to  ascertain  whether  a  transfer  of 
training  occurred  between  the  simulator  device  and  the  system. 
The  results  of  this  transfer  may  either  be  positive  or  nega¬ 
tive.  Positive  transfer  implies  that  less  time  is  needed  in 
the  aircraft  in  order  to  attain  a  predetermined  performance 
criterion  as  a  result  of  training  in  the  simulator,  while 


58 


*9asea»» 


negative  transfer  indicates  that  more  aircraft  time  is  required 
than  would  have  been  necessary  if  the  simulator  were  not  used 
in  the  training  process.  The  methodology  of  data  gathering 
would  require  that  experimental  groups  be  exposed  to  a  pre¬ 
designed  simulator  syllabus  and  then  performance  would  be 
measured  in  the  aircraft,  while  the  control  group  would  re¬ 
ceive  their  training  only  in  the  aircraft  before  completing 
the  performance  measurement. 

It  is  the  purpose  of  this  Section  to:  (1)  describe  some 
methods  that  have  been  developed  to  measure  the  effectiveness 
of  the  flight  simulator;  (2)  discuss  critical  factors  influ¬ 
encing  simulator  training  effectiveness;  and  (3)  present  an 
analysis  of  training  effectiveness  data  from  actual  military 
applications. 

B.  METHODS  OF  MEASURING  TRAINING  EFFECTIVENESS 

Computational  formulas  for  measuring  training  effective¬ 
ness  have  been  developed  using  the  relationship  between 
simulator  substitution  hours  and  in-flight  hours.  Table  IV- 1 
describes  how  the  methods  are  computed. 

Interpretation  of  the  calculations  indicates  that  the 
larger  the  positive  value  of  syllabus  reduction,  the  more 
effective  the  simulator  system,  and  the  smaller  the  Flight 
Substitution  Ratio.  (FSR) ,  the  more  effective  the  substitution. 
FSR  defines  the  rate  at  which  flight  time  is  being  replaced 
by  the  simulator  and  thus  reflects  efficiency  of.  the  device. 
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TABLE 


Transfer  Effectiveness  Ratio  (TER)  describes  the  ratio  of 
flight  hours  saved  to  the  time  spent  in  the  simulator  [Ref. 

59] .  All  the  flight  and  simulator  hours  data  used  in  the 
three  equations  are  the  times  required  for  the  subject  pilots 
to  accomplish  a  predetermined  effectiveness  criterion. 

Studies  conducted  between  1967  and  1977  support  the 
fact  that  simulators  3how  positive  transfer  effects  to  the 
aircraft;  however,  there  were  wide  variations  in  the  effec¬ 
tiveness  of  different  simulators  and  of  the  same  simulator 
when  used  for  different  types  of  training  [Ref.  60],  The  TER 
is  a  measure  used  to  identify  the  type  of  task  for  which  the 
simulator  would  be  more  cost  effective  than  the  aircraft. 

Using  the  learning  curve  theory,  the  amount  of  improvement 
per  hour  of  training  is  expected  to  decrease  as  training  pro¬ 
gresses.  This  implies  that  the  effectiveness  of  a  simulator 
is  the  greatest  at  the  beginning  of  training  and  diminishes 
during  the  training  period.  Despite  this  diminishing  effec¬ 
tiveness,  it  is  cost  effective  to  use  the  simulator  up  to  the 
point  where  the  TER  equals  or  becomes  less  than  the  ratio  of 
simulator  to  aircraft  operating  cost  [Ref.  61] . 

Although  the  effectiveness  of  flight  simulators  has  been 
quantitatively  measured  using  the  formulas  previously  explained, 
factors  that  influence  the  effectiveness  have  received  little 
attention.  Part  C  of  this  Section  will  address  these  factors. 
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C.  FACTORS  INFLUENCING  SIMULATOR  TRAINING  EFFECTIVENESS 
Even  though  state-of-the-art  technology  has  allowed 
simulator  systems  to  be  developed  with  remarkable  realism, 
there  remains  much  to  be  learned  about  training  with  these 
devices.  There  is  no  quantifiable  data  to  support  some  of 
the  factors  believed  to  influence  simulator  training  effec¬ 


tiveness;  however,  where  inferences  can  be  made  and  supported 
by  a  consensus,  these  factors  must  be  carefully  considered 
by  those  responsible  for  simulator  design.  The  following 
paragraphs  discuss  selected  factors  which  influence  training 


effectiveness. 

1.  Design  of  Simulator  Systems 

Important  in  the  design  of  simulator  systems  is  the 
fidelity  of  the  system  itself  and  training  aids  incorporated 
such  as  instructor  station  displays,  playback,  performance 
recording,  freeze,  and  performance  measurement.  The  level  of 
fidelity  is  often  equated  to  physical  correspondence  between 
the  simulator  and  the  real  world.  Two  critical  areas  con¬ 
tributing  to  the  level  of  fidelity  are  visual  systems  and 
motion  bases. 

When  simulating  aircraft  environments,  some  areas  of 
training,  such  as  air-to-ground  weapons  delivery  and  air-to- 
air  combat,  require  visual  systems  for  effective  training. 

On  the  other  hand,  instrument  flying  would  not  require  such 
a  system.  This  theory,  along  with  other  factors  that  will 
be  discussed  in  this  section,  have  motivated  the  military  to 
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evaluate  procuring  simulator  systems  which  include  part-task 
trainers  designed  to  train  within  particular  areas  of  mili¬ 
tary  flying  or  at  different  levels  of  pilot  performance.  As 
an  example,  when  a  student  begins  flight  training,  a  full 
fidelity  simulator  system  could  actually  serve  to  confuse 
him  or  her  by  having  attributes  such  as  visual  and  motion 
systems  which  would  divert  the  student's  attention  from  the 
actual  transfer  responses  desired.  Conversely,  advanced 
training  requires  the  high  fidelity  systems  which  most  closely 
simulate  the  actual  aircraft  through  sight,  sound,  and  motion. 
Using  a  low  fidelity  system  at  this  point  in  the  training 
cycle  would  seriously  reduce  the  effectiveness  because  of  the 
inability  to  provide  distractions  that  would  be  experienced 
during  aircraft  operation  [Ref.  62].  In  addition,  motion 
bases  provide  a  significant  contribution  to  the  costs  associ¬ 
ated  with  procuring  a  simulator  system;  therefore,  extensive 
study  and  evaluation  should  be  completed  to  ensure  that  motion 
systems  are  incorporated  within  the  part- task  trainers  that 
will  result  in  the  most  effective  training. 

During  the  period  of  August  1976  through  March  1977, 
the  Air  Force  Human  Resources  Laboratory  (AFHRL)  conducted 
a  study  to  determine  how  simulator  training  with  a  full  six 
degree  motion  system  affected  air-to-ground  weapons  delivery 
training.  Twenty-four  recent  graduates  of  the  pilot  training 
program  were  divided  into  three  groups  of  eight  subjects 
each.  The  groups  were  ranked  equally  in  the  areas  of  flight 
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grades  and  flight  experience.  The  control  group  received  no 
simulator  training.  Experimental  Group  I  received  training 
with  a  motion  simulator  and  Experimental  Group  II* s  training 
was  in  a  simulator  with  no  motion.  The  control  group  received 
two  familiarization  flights  in  the  F-5B  aircraft  prior  to  the 
flights  for  score.  There  were  two  ordnance  carrying  flights 
with  nine  bombing  patterns  made  on  each  flight.  The  pilot 
had  six  bombs  on  the  aircraft  so  he  would  make  one  practice 
pattern  and  then  drop  two  bombs.  This  scenario  was  repeated 
for  the  10°,  15°,  and  30°  delivery  patterns.  The  experimental 
groups  received  eight  one-hour  sessions  in  the  simulator  prac¬ 
ticing  the  three  tasks.  Following  these  sessions,  the  exper¬ 
imental  groups  completed  the  two  familiarization  flights  and 
then  flew  the  same  score-taking  flights.  The  method  for  meas¬ 
uring  the  bombing  score  was  the  circular  error  probable  (CEP) , 
which  is  the  radius  of  a  circle  in  which  50  percent  of  the 
projectiles  would  be  expected  to  fall.  Table  IV- 2  lists  the 
observed  means  for  each  group  on  the  three  bomb  delivery  tasks 
in  the  aircraft. 

Additionally,  scoreable  bombs  were  defined  as  those 
with  a-  CEP  of  less  than  300  feet.  The  results  of  the  three 
bombing  tasks  showed  that  72  percent  of  the  control  group's 
bombs  were  scoreable  while  86  percent  and  85  percent  of 
Experimental  Groups  I  and  II  were  scoreable. 

Statistically,  there  were  no  significant  differences 
found  between  the  two  experimental  groups  in  the  number  of 
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qualifying  bombs,  the  number  of  scoreable  bombs,  or  the 
bomb  delivery  circular  error.  These  results  indicated  that 
six-degree-of- freedom  platform  motion  did  not  enhance  the 
training  value  of  the  system.  To  reemphasize  the  fact  that 
the  level  of  fidelity  needed  within  the  system  should  be 
carefully  evaluated,  it  should  be  noted  that  the  aircraft 
simulated  was  a  T-37  (with  sighting  device)  and  still  there 
was  a  significant  transfer  of  training  to  the  F-5B.  This 
study  confirmed  the  theory  developed  by  Prophet  and  Boyd  in 
1970  that  a  low  fidelity  device  could  provide  considerable 
transfer  of  training  when  properly  utilized  [Ref.  63]. 

2.  Training  Programs 

Even  though  sophisticated  simulator  systems  have  been 
procured  by  the  military,  reports  have  been  documented  in 
which  the  importance  of  training  program  design  has  been  ig¬ 
nored  allowing  simulators  to  be  misused  or  used  inefficiently 
[Ref.  64].  Training  programs  should  be  implemented  that 
interface  with  the  simulator  devices.  A  dynamic  flight  sim¬ 
ulator  used  as  a  Cockpit  Procedures  Trainer  (CPT) ,  for  example, 
is  not  being  used  effectively.  A  procedure  that  has  proven 
to  be  most  effective  is  to  develop  and  utilize  training  pro¬ 
grams  that  are  presented  in  the  context  of  a  simulated  mission 
activity  as  opposed  to  an  abstract  training  exercise.  Litera¬ 
ture  has  also  shown  that  skills  taught  within  such  a  training 
program  were  retained  to  a  greater  degree  than  the  abstract 
training  exercise  [Ref.  65]. 
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3.  Personnel 


Trainees  and  instructors  both  play  a  significant 
role  in  the  training  effectiveness  of  flight  simulators, 
particularly  in  the  areas  of  qualifications  and  prior 
experience . 

As  was  highlighted  by  commercial  airlines’  use  of 
individualized  programs,  all  investigations  of  human  learn¬ 
ing  are  subject  to  the  influences  of  task-related  aptitudes 
of  the  trainees.  When  a  fixed  amount  of  time  is  allotted, 
trainees  with  a  high  aptitude  are  able  to  transfer  more 
training  to  the  aircraft;  however,  where  training  is  to  a 
fixed  performance  level  and  training  time  is  allowed  to  vary, 
both  high  and  low  aptitude  students  attain  this  performance 
level.  The  ability  of  a  simulator  to  train  less  experienced 
military  pilots  has  been  questioned;  however,  studies  have 
shown  that  the  flight  training  devices  and  programs,  if 
optimally  designed,  are  effective  for  pilots  with  differing 
experience  levels  [Ref.  66]. 

Training  effectiveness  of  a  simulator  system  is  highly 
dependent  upon  the  instructor's  input  into  the  program. 

Effectiveness  can  be  degraded  considerably  if  instructors 
are  not  fully  aware  of  the  capabilities  and  limitations  of 
the  system.  In  a  study  sponsored  by  the  Aerospace  Medical 
Laboratory,  it  was  determined  that  instructor  ability  and 
fidelity  of  simulation  were  related  in  such  a  way  that  as 

| 
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fidelity  increased  the  necessary  level  of  instructor  ability 
could  decrease,  and,  conversely,  as  fidelity  decreased  in¬ 
structor  ability  should  increase.  Unfortunately,  observations 
had  shown  that  just  the  opposite  occurred,  resulting  in  a 
reduction  of  the  simulator's  training  effectiveness  [Ref.  67]. 

A  final  personnel  point  to  support  training  effec¬ 
tiveness  is  the  single  instructor  concept.  Even  though  this 
variable  has  not  been  adequately  studied,  there  appears  to  be 
an  increase  in  effectiveness  when  a  single  instructor  is 
responsible  for  both  simulator  and  aircraft  training,  allowing 
instruction  given  in  the  simulator  to  be  more  compatible  with 
that  gi‘  \  in  the  aircraft.  This  reduce  j  any  potential  nega¬ 
tive  transfer  that  could  occur  as  a  result  of  instructor- 
peculiar  performance  requirements  [Ref.  68]. 

4 .  Expectations 

What  an  instructor  and  trainee  expect  from  a  simula¬ 
tor  system  can  affect  the  training  effectiveness.  If  simula¬ 
tors  are  viewed  as  useful  only  as  procedures  trainers  or  as 
instrument  trainers,  they  tend  to  be  used  only  in  that 
capacity,  even  though  possibly  offering  a  greater  range  of 
training  opportunities.  Attitudes  of  the  personnel  can  lead 
to  certain  expectations.  As  an  example,  it  was  noted  that 
older  pilots  tended  to  make  less  effective  flight  instructors, 
possibly  because  of  a  hesitancy  to  adopt  new  teaching  methods 
such  as  the  use  or.  simulation.  This  hesitancy  could  have 
occurred  as  a  result  of  unsatisfactory  experiences  with  older 
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simulators,  resulting  in  greater  confidence  in  in-flight 
training.  These  expectations,  especially  of  instructors, 
must  be  realized  before  training  effectiveness  of  the  system 
is  possible  [Ref.  69]. 

Effective  simulator  training  is  dependent  upon  a 
proper  combination  or  the  system,  training  programs,  per¬ 
sonnel,  expectations,  and  other  factors.  These  factors, 
which  may  not  be  that  influential  in  isolation,  may  become 
very  volatile,  either  positively  or  negatively,  when  acting 
in  combination.  Information  about  the  simulator's  design, 
the  way  it  was  used,  and  the  attitudes  and  expectations  of 
the  personnel  involved  should  be  disseminated,  resulting  in 
greater  benefits  to  simulator  training  programs  under  devel¬ 
opment.  The  first  step  in  this  process  in  to  recognize  a 
need  for  better  communication  amon^f  the  ufiers  of  simulator 
systems  which  would  lead  to  increased  training  effectiveness 
through  greater  familiarity  with  the  processes  [Ref.  70]. 

Measuring  the  training  effectiveness  of  the  flight 
simulators  in  military  scenarios  nas  been  performed  through 
many  independent  research  analyses.  The  next  part  of  Section 
IV  will  describe  the  results  in  the  mission  areas  of;  (1) 
instrument  flying  in  undergraduate  pilot  training,  (2)  air- 
to-surface  weapons  delivery  for  the  A-10  aircraft,  (3)  air- 
to-air  combat  training,  and  (4)  night  carrier  landing  training 
in  the  A-7E. 
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D.  TRAINING  EFFECTIVENESS  REALIZED  USING 

MILITARY  FLIGHT  SIMULATORS 

1.  Instrument  Flying  In  Undergraduate  Pilot  Training 

The  study  used  as  a  reference  for  this  section  was 
evaluating  the  simulator  system  mix  that  should  be  utilized 
in  the  instrument  training  program;  however,  information 
made  available  is  pertinent  to  demonstrating  training  effec¬ 
tiveness  of  such  devices  in  teaching  undergraduate  pilots 
the  mission  of  instrument  flying. 

The  study,  coordinated  by  AFHRL,  was  conducted  during 
the  period  of  March  1976  through  July  1977.  During  this 
period,  subject  pilots  were  divided  into  three  groups.  The 
first  group  received  all  instrument  training  in  the  aircraft. 
The  second  group  received  all  ground  instrument  training  in 
the  Advanced  Simulator  for  Pilot  Training  (ASPT)  and  all  pro¬ 
cedures  training  in  the  existing  T-37  instrument  procedures 
trainer  (T-4,  no  visual  and  limited  motion) .  The  third  group 
conducted  most  of  the  instrument  training  in  the  ASPT  with 
the  remainder  of  the  instrument  training  and  all  of  the  pro¬ 
cedures  training  in  the  T-4.  It  is  not  the  author's  intent 
to  discuss  the  proper  mix  of  these  two  trainers  as  recent 
phone  conversations  with  the  Instrument  Flight  Simulator  per¬ 
sonnel  at  Williams  AFB,  Arizona,  have  pointed  out  that  the 
T-4  is  presently  not  fully  operational  and  is  only  being  used 
as  a  Cockpit  Orientation  Trainer  (COT).  The  important  fact 
to  highlight  is  that  data  were  obtained  to  compare  hours  of 
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training  between  the  simulator  and  the  aircraft  and  also 
compare  the  average  "checkride"  scores  that  were  realized 
by  the  subjects  at  the  end  of  their  training  periods.  Al¬ 
though  the  group  mean  scores  for  the  T-37  "checkrides"  varied 
somewhat  between  the  experimental  and  control  groups,  none  of 
the  differences  were  statistically  significant  at  the  90  per¬ 
cent  confidence  level.  It  should  be  noted  that  the  control 
groups  received  an  average  of  11  instrument  training  sorties 
in  the  aircraft  while  the  experimental  group  received  an 
average  of  1.9  sorties  [Ref.  71].  Table  IV-3  outlines  the 
average  simulator  hours  and  aircraft  hours  used  by  the  exper¬ 
imental  and  control  groups  as  well  as  their  respective  average 
"checkride"  scores. 


TABLE  IV-3 

COMPARISON  OP  EXPERIMENTAL  AND  CONTROL 
GROUP  FOR  INSTRUMENT  STAGE  TRAINING 
(Source:  AFHRL  Study  TR-77-61) 


Control 

Experimental 

Group 

Group 

Average 

Simulator  Hours  Used 

17.3 

28.0 

Average 

Aircraft  Hours  Used 

15.8 

2.5 

Average 

"Checkride"  Scores 

90.52 

87.32 

Based  on  the  data  obtained,  it  was  determined  that 
use  of  flight  simulators  could  be  very  effective  in  instrument 
training.  The  present  syllabus  within  Air  Force  undergraduate 
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pilot  training  requires  22.1  hours  in  the  simulator  and  5.2 
hours  in  the  aircraft  for  Instrument  Phase  in  the  T-37  and 
31.2  hours  in  the  simulator  with  5.6  hours  in  the  aircraft 
for  Instrument  Phase  in  the  T-38.  Even  with  this  syllabus 
description,  the  average  percent  of  pilots  that  qualify  on 
their  first  instrument  "checkrides"  has  been  85  to  90  percent 
supporting  simulator  training  effectiveness  within  this 
mission  [Ref.  72]. 

2 .  Air-to-Surface  Weapons  Delivery  for  the 

A-10  Aircraft 

During  the  period  that  Gray  and  Fuller  were  conducting 
their  transfer  of  training  studies  from  the  T-37  simulator  to 
the  F-5B  aircraft,  the  A-10  aircraft  was  being  introduced  into 
the  Air  Force  inventory.  This  is  a  single-place  airplane  with 
a  primary  mission  of  air-to-surf ace  weapons  delivery;  there¬ 
fore,  simulator  training  was  especially  critical  due  to  the 
fact  that  the  first  flight  must  be  a  successful  solo  and  the 
flights  are  composed  of  hazardous  activities. 

Since  there  were  no  A-10  simulators  in  the  inventory, 
the  Tactical  Air  Command  (TAC)  and  AFHRL  modified  the  ASPT 
to  the  A-10  configuration.  This  allowed  two  objectives  to 
be  achieved.  First,  it  provided  AFHRL  the  opportunity  to 
further  their  research  in  air-to-surf ace  weapons  delivery 
training  with  a  simulator,  and  second,  it  provided  the  A-10 
neophyte  pilot  with  a  training  device  for  conversion  and 
surface  attack  tasks  [Ref.  73]. 
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It  was  recognized  from  the  beginning  that  the  sample 
size  would  be  small  because  of  the  recent  introduction  of  the 
aircraft;  however,  this  drawback  was  outweighed  by  the  valid¬ 
ity  of  the  sample.  The  subjects  were  very  representative  of 
the  population  because  both  the  students  and  instructors  were 
members  of  an  operational  combat  crew  training  squadron.  It 
was  determined  that  assessment  of  the  results  of  the  study 
would  be  based  on  measures  highly  relevant  to  actual 
operations  [Ref.  74] . 

The  modification  of  the  ASPT  to  the  A- 10  configura¬ 
tion  was  very  precise  in  the  area  of  aircraft  performance 
and  handling  qualities.  Although  the  cockpit  did  not  have 
complete  A- 10  instrumentation,  all  the  instruments  operated, 
and  the  CGI  system  depicted  Davis-Monthan  AFB  and  Gila  Bend 
Gunnery  Range  with  reasonable  fidelity  [Ref.  75] . 

The  study  consis.  :„d  of  six  air-to-surface  weapons 
delivery  tasks  including  dropping  bombs  at  different  dive 
angles  and  firing  the  internally-mounted  gun.  Bomb  scores 
were  determined  using  the  CEP  as  the  criterion  measure  while 
the  percentage  of  hits  per  rounds  fired  was  used  to  score  the 
firing  of  the  gun. 

There  were  24  students  participating  in  the  study. 

The  experimental  group  {17  students)  received  an  average  of 
six  hours  of  training  in  the  simulator  prior  to  flying  the 
aircraft.  The  control  group  (7  students)  did  not  receive  any 
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practice  in  the  simulator  before  flying  the  aircraft.  There 
were  12  sorties  flown  in  the  surface  attack  weapons  delivery 
phase.  The  first  seven  sorties  were  conventional  deliveries 
with  30°  dive  bomb  (DB) ,  20°  low  angle  low  drag  (LALD) ,  and 
15°  low  angle  bomb  (LAB) .  The  remaining  five  sorties  included 
5°  low  angle  strafe  (LAS)  and  "Pop-up"  deliveries  using  20° 
LALD  and  15°  LAB  [Ref.  76] .  "Pop-up"  deliveries,  as  opposed 
to  conventional  deliveries,  means  ingressing  into  the  target 
area  at  a  low  altitude,  pulling  the  nose  of  the  aircraft  up 
to  a  predetermined  pitch  attitude,  climbing  to  visually 
acquire  the  target,  and  then  completing  the  ordnance  delivery. 
Figures  IV-1  through  IV-5  graphically  portray  the  average  CEP 
scores,  in  meters,  that  were  realized  on  each  sortie  for 
bombing  tasks,  and  Figure  XV- 6  plots  the  percentage  of  rounds 
through  the  target  for  the  low  angle  strafe  task. 

The  data  made  available  to  this  author  were  only  the 
average  circular  error  probables  for  each  sortie.  To  support 
the  fact  that  all  tests  were  run  at  the  5  percent  level  of 
significance,  this  author  applied  a  t-test  of  statistical 
significance  to  the  average  data  to  compute  the  Prob-value 
(PV)  for  each  task.  The  null  hypothesis  (H  :  y.  -  y,  =  0) 
implied  no  difference  between  the  control  group  and  the 
experimental  group  while  the  alternative  hypothesis 
(K1  :  Vi  “  U2  >  0)  implied  that  the  average  CEP  for  the 
control  group  was  greater  than  the  experimental  group.  Table 
IV-4  outlines  the  t-statistic  value  and  the  Prob-value  for 


74 


FIGURE  IV-1 
30’  DIVE  BOMB 
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FIGURE  IV -2 
20*  LALO  DIVE  BOMB 
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FIGURE  IV-3 
15*  LAB  DIVE  BOMB 


FIGURE  IV -4 
20*  LALD  "POP-UP" 
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FIGURE  IV-5 
15*  LAB  "POP-UP" 


EXPERIMENTAL 

CONTROL 


FIGURE  IV-6 
5*  LOW  ANGLE  STRAFE 
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each  task.  It  must  be  remembered  that  the  values  were  cal¬ 
culated  from  average  scores  and  are  not  as  valid  as  raw  data. 
Since  the  PV  for  each  task,  except  the  20°  LALD  "Pop-up,"  is 
less  than  .05,  the  null  hypothesis  (HQ)  should  be  rejected 
at  the  95  percent  confidence  level,  meaning  that  there  is  a 
statistically  significant  difference  between  the  control 
group  and  the  experimental  group. 

TABLE  IV- 4 

t-STATISTIC  AND  PROB-VALUES 
FOR  A- 10  AIR- TO- SURFACE  TASKS 

Task  t-Statistic  (ts)  Prob-Value  (PV) 


30°  Dive  Bomb 

2.198 

.024 

20°  LALD  Dive  Bomb 

4.774 

.0002 

15°  LAB  Dive  Bomb 

2.63 

.013 

20°  LALD  "Pop-up" 

-3.78 

.997 

15°  LAB  "Pop-up" 

1.910 

.046 

5°  LAS 

3.929 

.002 

As  depicted  on  Figure  IV- 4  and  supported  by  the 
average-based  t-statistic  and  PV  in  Table  IV-4,  the  control 
group  exhibited  significantly  better  performance  than  the 
experimental  group  in  the  20°  LALD  "Pop-up"  task.  It  was 
determined  that  ASPT's  flat  visual  flight  plane  offered 
limited  depth  perception  cues.  As  a  result,  the  ability  to 
judge  angle-off  approaches  and  apex  positioning  made  the 
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"Pop-upu  weapons  delivery  task  more  difficult  to  satisfac¬ 
torily  simulate  [Ref.  77]. 

In  conclusion,  it  was  determined  that  three  ASPT 
sorties  (6  hours)  provided  highly  effective  training.  The 
data  indicated  that  the  simulator  also  provided  more  bene¬ 
ficial  training  in  the  conventional  delivery  and  low  angle 
strafe  than  in  the  "Pop-up"  delivery.  Additional  benefits 
included  flight  safety  and  ease  of  instruction  on  the  con¬ 
trolled  range.  Mechanical  and  procedural  problems  could  be 
either  eliminated  or  reduced  in  the  simulator,  leaving  more 
time  for  instructing  weapon  delivery  techniques  on  the  actual 
range  [Ref.  78]. 

3 .  Air-to-Air  Combat  Training 

State-of-the-art  simulators  provide  an  excellent 
opportunity  to  enhance  training  effectiveness  in  air-to-air 
combat.  If  the  aerodynamic  capabilities  are  properly  pro¬ 
grammed,  it  allows  the  pilot  an  opportunity  to  explore  the 
edges  of  the  maneuvering  envelope  without  sacrificing  safety 
of  flight.  It  also  gives  the  pilot  excellent  training  in 
utilizing  different  weapons  systems  such  as  missiles  and  guns 
and  also  to  develop  both  offensive  and  defensive  tactics. 

The  U.S.  Air  Force,  through  a  joint  program  with  the 
Vought  Corporation,  has  developed  a  Good  Stick  Index  (GSI) 
for  measuring  the  training  effectiveness  of  air-to-air  combat 
simulation.  The  GSI  was  formulated  from  data  gathered  in 
Vought' s  Air  Command  Engagement  Simulator  (ACES).  The  ACES 
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consists  of  two  cockpits,  each  situated  within  16  foot 
diameter  spherical  screens.  It  utilizes  F-4  cockpits  with 
complete  instrumentation  plus  the  ability  to  program  into 
the  computer  MIG-21  aerodynamic  characteristics  in  order  to 
provide  training  in  dissimilar  aircraft  engagements  [Ref.  79] . 

Normally,  classes  utilizing  the  ACES  Training  Program 
consisted  of  eight  students.  The  students  would  fly  approxi¬ 
mately  8.5  to  9.0  hours  in  the  simulator  between  Monday  and 
Friday.  The  study  discussed  in  this  part  of  Section  IV  exam¬ 
ined  data  from  four  of  twelve  classes  which  used  the  simulator 
from  3  April  1978  through  23  June  1978.  During  this  period, 

89  students  were  evaluated.  At  the  end  of  the  training 
period,  a  "turkey  shoot"  tournament  was  conducted  in  which 
the  students  flew  one-versus-one  engagements  with  double 
elimination  rules.  It  was  the  intent  of  the  study  to  statis¬ 
tically  validate  the  GSI  as  a  predictor  of  the  "turkey  shoot" 
winner,  investigate  improvement  in  the  GSI  by  varying  the 
weight  factors  of  the  parameters,  and  introduce  additional 
parameters  to  determine  whether  the  predictions  could  be  im¬ 
proved.  These  objective  evaluations  were  also  compared  with 
predictions  of  the  instructor  pilots  to  assess  its  agreement 
with  expert  opinion  [Ref.  80].  (Readers  interested  in  further 
details  concerning  the  study  or  the  statistical  analysis  are 
encouraged  to  read  AFHRL-TR-79-15  which  may  be  obtained  from 
the  Air  Force  Human  Resources  Laboratory,  Air  Force  Systems 
Command,  Brooks  AFB,  Texas.)  This  author  intends  to  define 
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the  GSI  and  show,  through  the  data  gathered  from  the  four 
classes,  that  the  pilot's  ability  did  improve  during  the  one 
week  of  training. 

Computation  of  the  GSI  score  was  accomplished  using 
data  obtained  from  the  predictor  variables  measured  by  the 
computer  system.  The  data  were  recorded  when  the  simulator 
was  flown  against  five  canned  targets.  The  five  canned  tar¬ 
gets  consisted  of  two  cinetrack  runs  and  three  head-on  passes. 
The  coefficients  of  the  equation  were  computationally  weighted 
to  provide  an  ideal  score  of  1000.  The  equation  developed  is, 

GSI  =  4.6(70  -  MILERR)  +0.86  (PANG)  +  (O/D  -  35) 

+0.5  (180  -  TTFK) 

where: 

MILERR  =  average  mill  error  over  two  cinetrack  runs 
while  range  is  less  than  3000  feet. 

PANG  =  average  percentage  of  engagement  time  in 

pointing  angle  advantage  over  two  cinetrack 
runs  at  a  range  less  than  3000  feet. 

O/D  =  average  ratio  of  offensive  to  defensive  time 
against  the  head-on  targets.  Offensive 
time  is  the  time  the  target  aircraft  is  in 
the  front  hemisphere  of  the  piloted  aircraft. 

TTFK  =  average  time  to  first  kill  (seconds)  from 

beginning  of  run  until  student  achieves  first 
kill  against  head-on  targets  with  gun  or  heat 
missile.  [Ref.  81] 

Table  IV- 5  lists  the  Monday  and  Friday  GSI  scores 
for  27  pilots  tested  during  this  study.  Hypothesis  testing 
is  used  to  compute  the  t-statistic  and  the  PV  from  the  actual 
data.  The  null  hypothesis  is  that  there  is  no  difference  in 
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TABLE  IV- 5 


pitot  gsi  s-oke'  before  and  after  simulator  training 

PILOT  GSI  *  ,ujrce.  study  TR-79-15) 


Pilot  l.o 

1 

2 

3 

4 

5 

6 

7 

8 
9 

1  0 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 
27 


After  Training 
(Friday) 


Before  Training 
(Monday) _ 


595 

601 

589 

547 

499 

549 

552 

794 

447 

562 


359 

312 

266 

125 

309 

393 

304 

210 

531 

234 


570 

494 

487 

851 

739 

751 

531 

527 

716 

581 


304 

199 

393 

687 

391 

553 

247 
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the  GSI  between  Monday  and  Friday  versus  the  alternative 
hypothesis  that  there  exists  a  statistically  significant 
difference.  Since  the  PV  is  less  than  .05,  we  may  reject 
the  null  hypothesis  that  the  population  means  are  equal  and 
accept  the  alternative  hypothesis  that  the  Friday  GSI  score 
is  greater  than  the  Monday  score. 

Through  the  ACES,  the  Tactical  Air  Command  has  been 
able  to  complement  and  reinforce  flight  training  in  the  air- 
to-air  environment.  The  program  has  been  developed  to  opti- 
mizfc  simulation  training,  so  that  it  will  enhance  the  flight 
syllabus,  not  replace  it.  The  simulator  system  involves  all 
facets  of  one-versus-one  similar  and  dissimilar  training, 
including  the  ability  to  train  with  all  ordnance  systems. 

The  debriefing  ability  .includes  data  recording,  replay,  and 
video  recording.  The  scoring  system  looks  at  ordnance  firing 
conditions  and  combat  management.  Every  tenth  bullet  is 
scored  and  the  attacker's  "g"  and  angle  of  attack  are  recorded 
along  with  the  aiming  error,  range,  angle  off,  and  target 
crossing  angle.  Missile  firing  data  is  evaluated  to  determine 
if  the  missile  was  fired  within  the  proper  envelope,  and,  if 
not,  which  parameters  were  not  satisfied. 

The  cost  of  operating  the  simulator  is  approximately 
$250  per  hour  per  cockpit  compared  to  the  operating  cost  of 
the  F-4  which  is  about  $1500  per  hour.  These  values  take  on 
much  greater  meaning  when  one  considers  that  during  a  one 
hour  flight  a  pilot  would  experience  approximately  three 
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engagements  in  the  aircraft  and  15  engagements  in  the  simu¬ 
lator.  During  the  one  week's  training  a  pilot  will  receive 
an  average  of  134  engagements  [Ref.  82] . 

General  Robert  Dixon,  Commander  of  the  Tactical  Air 
Command,  reemphasized  the  value  of  air-to-air  simulation 
training  when  he  stated. 

With  today's  advanced  state-of-the-art  in  lethal  air 
combat  weapons  systems,  TAC  feels  that  training  must  be 
better  than  ever  before  so  that  every  advantage  of  the 
system  over  those  of  the  enemy  may  be  fully  exploited 
by  the  key  ingredient  of  the  system,  the  man.  TAC  must, 
through  optimized  training,  make  the  combat  pilot's 
capability  match  that  of  his  equipment.  His  appraisals, 
decisions,  and  actions  must  match  the  speed  and  accuracy 
of  his  machine.  And,  in  these  austere  times,  the  Air 
Force  is  particularly  interested  in  improved  pilot 
capability  without  an  appreciable  increase  in  cost. 
fRef.  83]. 

This  improved  pilot  capability  can  be  realized  through 
effective  utilization  of  an  air-to-air  combat  simulator. 

4 .  Night  Carrier  Landing  Training  in  the  A-7E 

Training  effectiveness  of  the  A-7E  Night  Carrier 
Landing  Trainer  (NCLT)  was  determined  by  using  pilots  that 
were  members  of  the  Replacement  Air  Group  (RAG)  squadron. 
The  pilot  trainees  were  selected  for  NCLT  training  (exper¬ 
imental  Group)  or  NO-NCLT  training  (control  group)  on  the 
basis  of  the  following  specific  selection  and  assignment 
criteria; 

(a)  No  previous  A-7E  experience. 

(b)  No  night  carrier  landing  experience  in  any 
aircraft  for  the  last  three  years. 
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(c)  Random  selection  from  newly  designated  aviators, 
pilots  who  had  instructed  in  the  training  command 
but  with  no  fleet  experience,  pilots  who  had  been 
assigned  to  billets  with  variable  flying  time  for 
the  past  18  months  to  three  years,  and  ex-prisoners 
of  war  or  experienced  aviators  who  had  not  flown 

in  two  or  three  years. 

(d)  Random  selection  of  pilots  on  the  basis  of  previous 
jet  flight  hours. 

From  these  criteria,  26  pilots  were  chosen  for  NCLT  training 
while  27  pilots  were  placed  in  the  NO-NCLT  training  group. 
Their  average  jet  flight  hours  were  715  and  74.7  hours 
respectively  [Kef.  84]. 

The  planned  simulator  training  schedule  provided  for 
the  experimental  group  to  receive  6.5  hours  of  training  with 
approximately  85  controlled  approaches.  The  control  group 
received  2.5  hours  of  familiarization  training  in  the  simu¬ 
lator  but  no  controlled  approaches  (Ref.  85] . 

There  were  both  objective  carrier  landing  performance 
measures  and  subjective  performance  measures  used  to  determine 
the  transfer  of  training  effectiveness.  Variables  such  as 
altitude  and  lateral  error  from  glideslope  and  centerline,  an 
objective  measure  derived  from  wire  number  arrestment ,  and 
the  percentage  of  final  approaches  that  resulted  in  successful 
landings  were  some  examples  of  the  objective  landing  perfor¬ 
mance  measurements,  while  the  Landing  Signal  Officer  (LSO) 
evaluations  and  pilot  questionnaires  provided  the  qualitative 
performance  measurements  [Ref.  86]. 
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Both  the  quantitative  and  qualitative  performance 
measurements  had  statistical  tests  applied  to  their  values. 

The  Night  Carrier  Qualification  Landing  Performance  Score 
was  significantly  different  for  NCLT  pilots  (ts  =  1.98, 
df  =  46,  PV  =  .02).  The  night  boarding  rate  was  also  sig¬ 
nificantly  different  for  the  NCLT  pilots  based  on  a  t-test 
of  proportions  (ts  =  1.643,  df  =  46,  PV  =  .05).  The  NCLT 
group  (185  approaches)  had  a  7  percent  higher  boarding  rate 
than  the  NO- NCLT  group  (202  night  approaches) .  The  overall 
attrition  rate  was  found  to  be  significantly  different 
between  the  two  groups  (ts  =  2.50,  df  =  51,  PV  =  .006).  The 
NCLT  pilots  had  one  disqualification  out  of  26,  while  there 
were  eight  disqualifications  out  of  the  27  NO- NCLT  pilots 
[Ref.  87] .  This  means  that  96  percent  of  the  NCLT  trained 
pilots  were  successful  in  their  first  carrier  qualification 
period.  On  the  other  hand,  disqualified  pilots  averaged  12 
weeks  longer  in  training  plus  they  required  19  percent  more 
total  A-7E  flight  hours  and  33  percent  more  night  hours  be¬ 
fore  night  carrier  qualification  [Ref.  88],  In  terms  of 
qualitative  measures,  LSO  evaluations  of  final  approach  and 
carrier  landings  were  found  to  be  statistically  different  at 
night  (ts  =  2.78,  df  =  46,  PV  “  .003)  [Ref.  89]. 

It  was  also  determined  from  analyzing  the  data  that 
the  recently  designated  aviator  received  very  effective 
training  from  the  NCLT.  His  attrition  rate  was  only  8  percent 
while  the  new  aviator  that  did  not  train  in  the  NCLT  had  an 


86 


attrition  rate  of  44  percent.  New  aviators,  with  NCLT 
training,  had  a  50  percent  success  rate  for  their  first 
night  approach  compared  to  a  31  percent  success  rate  for 
the  new  aviator  with  NO-NCLT  experience  [Ref.  90]. 

Data,  such  as  described  in  the  preceding  paragraphs, 
support  the  theory  that  the  NCLT  is  an  effective  trainer  for 
preparing  aviators  for  night  carrier  qualification.  From  a 
safety  standpoint,  the  night  carrier  accident  rate  has  been 
reduced  by  80  percent  since  introduction  of  the  NCLT  into 
the  training  process  (Ref.  91] . 

E.  SUMMARY 

Section  IV  has  described  methods  of  measuring  training 
effectiveness,  factors  influencing  simulator  training  effec¬ 
tiveness,  and  training  effectiveness  realized  from  military 
flight  simulators.  No  studies  examined  by  this  author  would 
support  a  finding  that  flight  simulators  do  not  provide 
effective  training;  however,  improving  the  system's  cost 
effectiveness  requires  further  examination.  Section  V  will 
investigate  the  areas  of  improving  and  measuring  cost  effec¬ 
tiveness  of  visual  flight  simulation. 
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V.  COST-EFFECTIVENESS  OF  MILITARY  FLIGHT  SIMULATORS 


A.  GENERAL 

In  a  report  recently  released  by  Frost  and  Sullivan, 
Incorporated,  it  was  disclosed  that  spending  on  flight  sim¬ 
ulators  and  training  devices  by  the  military,  NASA,  and 
civilian  markets  was  projected  to  total  slightly  below  $7 
billion  between  FY  1981  and  FY  1985.  The  Navy  was  forecast 
to  emerge  as  the  lar  jest  buyer  during  this  period  with  a 
projected  outlay  of  $2.44  billion,  equating  to  35  percent  of 
the  total  market  [Ref.  92] .  With  the  amount  of  money  being 
invested  in  flight  simulators,  it  is  obvious  that  these 
systems  have  become  an  integral  part  of  the  total  weapons 
system  procurement  for  the  military;  therefore,  it  is  abso¬ 
lutely  critical  that  the  systems  be  utilized  in  the  most 
cost-effective  manner.  Section  IV  described  the  training 
effectiveness  of  military  flight  simulators,  but  it  should 
be  noted  that  the  fact  that  flight  simulators  are  effective 
for  training  does  not  necessarily  imply  that  the  systems  are 
worth  their  cost  [Ref.  93], 

Section  V  will:  (1)  examine  a  method  that  has  been  used 
by  the  Navy  to  measure  the  cost-effectiveness  of  flight  simu¬ 
lators;  (2)  describe  an  application  of  microeconomic  theory 
that  could  approximate  the  optimum  mix  of  training  hours 
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between  an  aircraft  and  simulator,,  and  therefore  improve  cost- 
effectiveness;  (3)  outline  the  cost-effectiveness  analysis 
used  by  the  Army  for  their  AH-1  helicopter  flight  simulator; 

(4)  describe  a  model  developed  by  the  Analytic  Services 
(ANSER  MODEL)  for  determining  the  cost-effectiveness  of  air¬ 
crew  training  devices;  and  (5)  suggest  an  area  that  could  be 
studied  in  order  to  improve  the  cost-effectiveness  of  simu¬ 
lator  systems. 

B.  NAVAL  AIR  SYSTEMS  COMMAND  COST-EFFECTIVENESS  METHODOLOGY 

The  Navy  conducts  a  cost-benefit  analysis  as  part  of  the 
procedure  used  to  determine  whether  or  not  to  procure  a 
flight  simulator  system.  The  costs  associated  v/ith  the  anal¬ 
ysis  include  the  capital  investment,  any  cost  associated  with 
modifications  or  updates  to  the  system,  annual  operations  and 
maintenance  costs,  ar*u  ny  military  construction  necessary  to 
"house"  or  support  the  system.  Benefits  include  cost  savings 
associated  with  flight  hour  substitution,  depreciation  sav¬ 
ings,  and  the  assumed  savings  associated  with  accident 
reduction. 

Cost  savings  are  determined  by  using  the  following 
aquation: 

Cost  Savings  -  (Cost/Flight  Hour)  x 

(Flight  Hour  Substitution) 

Cost  per  flight  hour  for'  each  aircraft  can  be  obtained  from 
the  Navy  Program  Factors  Manual  and  includes  cost  associated 
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with  petroleum-oil-lubricants  (POL)  cost  per  hour,  organiza¬ 
tional  and  intermediate  maintenance  per  flight  hour,  component 
rework,  replenishment  spares,  and  engine  overhaul.  Plight 
hours  substitution  is  the  projected  number  of  hours  in  a  year 
that  can  be  substituted  in  the  simulator  instead  of  flown  in 
the  aircraft. 

Information  necessary  to  compute  depreciation  savings 
includes  aircraft  acquisition  cost,  aircraft  service  life, 
the  number  of  aircraft  at  the  base  where  the  simulator  is 
located,  and  the  number  of  flight  hours  projected  to  be  flown 
by  an  aircraft  during  the  year.  Depreciation  savings  are 
calculated  by  using  the  following  equation: 

Depreciation  Savings  =  (Depreciation/Aircraft)  x 

(Number  Aircraft)  x 

.Fit.  Hour  Substitution  . 

Fit.  Hrs  +  Fit.  Hrs.  Subs: 

The  third  area  measured  in  order  to  determine  annual 
benefits  is  accident  reduction.  The  assumption  used  to 
support  this  measurement  is  that  all  pilot  error  accidents 
could  have  been  avoided  by  using  flight  simulators.  An 
estimated  monetary  loss  for  damage  to  aircraft  due  to  pilot 
error  is  determined  and  then  multiplied  by  the  yearly  flight 
hour  substitution. 

Accident  Reduction  Savings  =  x  Fit.  Hr.  Subs. 

Other  factors  considered  in  procurement  of  a  simulator 
system  includes  the  value  of  a  human  life  and  the  emergency 


training  capability  which  is  impossible  or  unsafe  to  conduct 
in  the  aircraft.  Both  of  these  factors  are  difficult  to 
reduce  to  monetary  quantities  [Ref.  94]. 

Utilizing  the  cost-benefit  methodology,  this  author 
performed  an  analysis  using  data  from  the  Instrument  Flight 
Simulator  Training  Division  at  Williams  AFB,  Arizona.  De¬ 
preciation  savings  was  not  computed  since  the  aircraft  at 
Williams  (T-37  and  T-38)  had  exceeded  their  service  lives, 
making  depreciation  zero.  It  should  also  be  noted  that  in 
computing  the  accident  reduction  savings,  $260,000  was  used 
as  a  value  for  a  human  life  [Ref.  95] .  The  amount  shown  for 
initial  capital  investment  also  includes  modifications  and 
updates.  Table  V-l  presents  the  cost-effectiveness  analysis. 
Present  values  are  computed  using  a  10  percent  discount 
factor  as  required  by  Department  of  Defense  Instruction 
7041,3,  Economic  Analysis  and  Program  Evaluation  for  Resource 
Management. 

During  the  period  FY  78  through  FY  80,  the  net  benefit 
associated  with  the  instrument  flight  simulators  totaled 
$31,336,796,  which  means  that  the  system  was  fully  amortized 
in  slightly  less  than  three  years.  As  evidenced  by  the  values 
in  Table  V-l,  flight  hour  substitution  is  the  primary  benefit 
realized  from  flight  simulator  utilization;  however,  the 
substitution  hours  are  "best-guess"  subjective  estimates  and 
are  not  objectively  determined.  The  next  section  will  outline 
a  procedure  for  realizing  the  optimum  mix  of  simulator  and 

aircraft  hours  using  microeconomic  theory  as  a  framework. 
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TABLE  V— 1 

COST-BENEFIT  ANALYSIS  FOR  INSTRUMENT  FLIGHT  SIMULATORS 
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C.  IMPROVING  COST-EFFECTIVENESS  USING  MICROECONOMIC 
THEORY  AS  A  FRAMEWORK 

The  microeconomic  theory  being  used  is  normally  employed 
when  showing  production  with  two  variable  inputs.  An  example 
of  variable  inputs  would  be  labor  and  capital.  Different 
combinations  of  labor  and  capital  can  be  used  to  produce  the 
same  output.  Figure  V-l  depicts  this  by  plotting  an  isoquant 
on  a  graph  with  capital  on  the  y-axis  and  labor  on  the  x-axis. 


FIGURE  V-l 
PRODUCTION  ISOQUANT 


The  graph  shows  that  different  combinations  of  inputs  such 
as  50  units  of  capital  and  15  units  of  labor  or  10  units  of 
capital  and  75  units  of  labor  produce  the  same  ldo  units  of 
output;  therefore,  an  isoquant  is  a  curve  showing  all  possible 
combinations  of  inputs  physically  capable  of  producing  a  given 


93 


a 


level  of  output.  Isoquants  are  concave  from  above,  indicat¬ 
ing  a  diminishing  marginal  rate  of  technical  substitution. 
This  concavity  implies  that  as  capital  decreases  by  equal 
amounts,  proportionately  more  labor  must  be  added  in  order 
to  maintain  the  same  output  level  [Ref.  96]. 

With  the  isoquant  plotted,  the  producer  is  now  concerned 
with  the  costs  associated  with  the  two  inputs.  In  determin¬ 
ing  his  operating  input,  it  is  important  for  the  producer 
to  pay  particular  attention  to  relative  input  prices  in  order 
to  minimize  the  cost  of  producing  a  given  output  or  maximize 
output  for  a  given  level  of  cost. 

The  next  step  in  the  analysis  would  be  to  plot  a  cost 
constraint  on  a  graph  such  as  depicted  in  Figure  V-l.  As 
an  example,  suppose  the  cost  of  capital  was  $1000  per  unit 
and  labor  wage  rates  were  $2500  per  man  year.  If  the  deci¬ 
sion  was  made  to  invest  $15,000  in  the  two  inputs,  then  the 
producer  could  invest  the  total  amount  in  capital  resulting 
in  15  units  being  purchased,  or  he  could  invest  totally  in 
labor  allowing  him  to  purchase  six  man  years.  Using  these 
two  points,  the  producer  could  plot  the  cost  constraint,  line 
C  on  the  isoquant  graph  as  depicted  in  Figure  V-2. 

The  producer  would  use  the  combination  of  inputs  where 
the  cost  constraint,  line  C’ ,  is  tangent  to  the  production 
isoquant.  This  satisfies  the  principle  of  maximizing  output 
subject  to  a  given  cost  or  minimizing  cost  subject  to  a  given 
output  because  at  the  tcingency  point  the  marginal  rate  of 
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technical  substitution  is  equal  to  the  input  price  ratio 
(the  price  of  labor  to  the  price  of  capital) .  For  the  pro¬ 
ducer,  the  criterion  for  fixed  effectiveness  at  the  minimum 

cost  would  be  to  use  K  units  of  capital  and  L  units  of 

o  o 

labor  [Ref.  97] . 


FIGURE  V-2 

PRODUCTION  ISOQUANT  AND  COST  CONSTRAINT 


This  analysis  can  be  applied  to  the  determination  of  the 
optimum  mix  of  aircraft  hours  and  simulator  hours  necessary 
to  attain  a  required  level  of  effectiveness  at  the  minimum 
operating  cost.  In  1973,  a  study  was  conducted  by  Povenmire 
and  Roscoe  in  which  they  developed  an  effectiveness  isoquant 
by  plotting  the  average  hours  needed  for  students  to  pass 
their  final  flight  check  after  practicing  a  certain  number 


95 


of  hours  in  the  Piper  Cherokee  and  the  Link  GAT-1  trainer. 
Table  V-2  outlines  the  results  of  this  study  [Ref.  98] . 


TABLE  V-2 


SIMULATOR  HOURS  VS.  AIRCRAFT  HOURS 
HEEDED  TO  PASS  FINAL  FLIGHT  CHECK 
(Source:  Povenmire  and  Roscoe,  1973) 


Group 

Aircraft  only 

Simulator 
3  Hours 
7  Hours 
11  Hours 


Group  Size 
14 

13 

9 

10 


Average  Hours  Needed  to 
Pass  Final  Flight  Check 

45.4 


40.3 

38.6 

37.9 


If  one  were  to  plot  and  connect  the  four  points,  the 
resultant  curve  would  be  an  effectiveness  isoquant  because 
all  combinations  of  aircraft  and  simulator  time  result  in 
the  student  being  able  to  pass  his  final  flight  check,  i.e. , 
fixed  effectiveness.  The  next  step  in  the  procedure  would 
be  to  determine  the  operating  cost  per  hour  for  the  aircraft 
and  simulator.  By  taking  these  costs  and  dividing  each  into 
some  fixed  investment  dollars,  a  cost  constraint  line  could 
be  determined.  The  tangency  point  between  this  line  and  the 
effectiveness  isoquant  would  show  the  optimum  mix  of  aircraft 
hours  and  simulator  hours  needed  to  satisfy  the  fixed  effec¬ 
tiveness  at  the  minimum  cost. 


D.  COST-EFFECTIVENESS  ANALYSIS  FOR  THE  AH-1  HELICOPTER 

The  Army  conducted  a  cost  and  training  effectiveness 
analysis  (CTEA)  for  the  AH-1  "Cobra"  flight  simulator 
(AH-1FS)  and  released  it  through  the  Directorate  of  Training 
Developments,  U.S.  Army  Aviation  Center,  Fort  Rucker,  Ala¬ 
bama,  in  November  1979. 

The  cost-effectiveness  portion  of  the  study  was  divided 
into  three  major  areas,  including  the  task  training  analysis, 
the  cost  analysis,  and  the  task  priority. 

As  with  most  studies  described  in  this  thesis,  two  groups 
were  designated  to  undergo  the  required  training  with  the 
control  group  receiving  all  training  in  the  aircraft  and  the 
experimental  group  completing  simulator  training  before  fly¬ 
ing  the  aircraft.  All  aviators  in  both  groups  had  completed 
primary  rotary  wing  training  in  the  UH-1  "Huey"  helicopter. 

The  control  group  was  comprised  of  14  aviators  with  an  average 
cf  517  rotary  wing  hours  while  the  experimental  group  had  26 
aviators  with  an  average  experience  level  of  546  rotary  wing 
hours  [Ref.  99] . 

Data  were  collected  for  both  the  experimental  and  control 
group  for  44  separate  tasks.  The  data  forms  were  identical  so 
that  training  in  the  simulator  could  be  compared  to  training 
in  the  aircraft.  Examples  of  tasks  to  be  performed  included: 
hover  flight,  maximum  performance  takeoff,  running  landings, 
and  firing  of  rockets. 


the  task 


The  first  step  in  the  study  was  to  det :  • .  ■* 
training  anaJysis.  For  each  of  the  44  distincu  tasks,  a 
curve  was  developed  showing  the  relationship  between  task 
training  in  the  simulator  and  the  aircraft.  The  equation 

— K  Y 

for  this  curve  was  in  the  form  of  Y  =  b^e  2  +  where 

x  is  defined  as  the  amount  of  simulator  training  time  and  Y 
is  the  amount  of  additional  aircraft  training  time  required 
to  meet  the  standard.  The  variables  b^,  b2,  and  b^,  were 
numerical  values  determined  from  a  least  squares  fit  to  the 
collected  data.  The  fixed  effectiveness  utilized  was  deter¬ 
mined  by  Aviator  Qualification  Course  (AQC)  standards  for 
each  of  the  tasks.  For  the  task  of  running  landings,  the 
equation  for  the  curve  was  Y  =  5.6”°*32x  +  2.51  [Ref.  100] , 
The  curves  that  were  developed  for  each  task  were  also 
used  to  determine  the  most  cost-effective  mix  of  simulator 
and  aircraft  training  time.  Instead  of  using  the  graphical 
approach  as  previously  described,  the  analysts  used  calculus 
to  minimize  the  total  cost  equation.  This  equation  was  in 
the  form  of : 

Ct  =  C]Y  +  C2X 

where  =  total  task  training  cost 

Cx  =  aircraft  task  training  cost  per  minute 
C2  *  simulator  task  training  cost  per  minute 
X  =  simulator  task  training  time  (minutes) 

Y  =  aircraft  task  training  time  (minutes) . 

Substituting  Y  =  b^e  ^2X  +  b^  into  the  total  cost  equation 
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and  taking  the  derivative  of  this  equation  with  respect  to 
X,  the  mix  of  simulator  and  aircraft  time  needed  in  order 
to  meet  the  standard  at  the  lowest  cost  was: 


1  c1^1^2 
m  b2  LN  v  c 2  1 

Y  =  b  e~^2xm  +  h 
m  1  3 

where  X  and  Y  are  the  simulator  and  aircraft  training  times 
mm 

in  minutes.  These  training  times  were  calculated  for  each  of 
the  44  tasks.  In  the  case  of  running  landings,  the  optimum 
mix  was  calculated  to  be  5  minutes  in  the  simulator  and  an 
additional  4  minutes  in  the  aircraft  [Ref.  101] . 

Once  the  fixed  effectiveness/minimum  costs  training  times 
were  calculated,  the  cost  savings  associated  with  using  the 
simulator  and  aircraft  vice  the  aircraft  only  were  calculated. 
In  order  to  determine  the  cost  per  minute  of  the  simulator  and 
the  aircraft,  the  percent  of  time  utilized  by  the  device  for 
training  the  particular  task  was  used  as  part  of  the  calcula¬ 
tion.  As  an  example,  the  operating  and  support  cost  plus  the 
accident  cost  rate  per  minute  for  the  AH-1  was  $9.50  (FY  79 
dollars).  Historical  data  showed  that  73.4  percent  of  flight 

time  was  dedicated  to  the  task  training?  therefore,  the  cost 

eg  eg 

per  minute  would  be  ■  ^  =  $12.94.  The  cost  associated 
with  using  the  simulator  was  *  $4.98.  This  implied 

that  100  percent  of  simulator  time  was  dedicated  to  task 
training.  From  the  data,  the  total  cost  savings  for  each 
task  could  be  calculated  [Ref.  102], 
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After  calculating  the  cost  per  minute  for  the  simulator 
($4,98)  and  the  aircraft  ($12.94),  this  author  applied  the 
microeconomic  theory  previously  described  to  arrive  at  the 
optimum  mix  of  device  training  time  at  the  minimum  cost. 

Figure  V-3  shows  the  effectiveness  isoquant  (Y  =  5.6e""®*^x 
+  2.51)  that  was  developed  for  running  landings  with  the  cost 
constraint  and  tangency  point  showing  the  respective  times. 
Even  with  a  hand-drawn  curve  of  only  five  points,  the  point 
of  tangency  indicates  slightly  more  than  5  minutes  of  simu¬ 
lator  time  (Xm)  and  slightly  less  than  4  minutes  of  additional 

aircraft  time  (Y  ) . 

m 

The  final  step  in  the  cost-effectiveness  process  was  to 
priortize  the  tasks  in  order  of  their  dollar  savings.  Tasks 
with  the  highest  values  were  assigned  the  highest  priority. 
With  the  priortized  list  completed,  the  annual  time  available 
to  train  students  on  the  simulator  could  be  economically  allo¬ 
cated.  Information  such  as  the  number  of  students  expected 
during  the  next  year  and  the  number  of  simulator  training 
hours  available  could  be  used  to  calculate  the  number  of  hours 
each  student  could  receive  on  the  device.  With  the  student's 
training  time  calculated,  one  could  start  adding  the  simu¬ 
lator  minutes  as  he  went  down  the  priortized  list  until  the 
time  was  less  than  or  equal  to  the  student's  allotment.  At 
that  point,  the  dollar  savings  associated  with  each  accom¬ 
plished  task  could  be  totaled,  giving  the  cost  savings  per 
student. 
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FIGURE  V-3 

ISOQUANT  AND  COST  CONSTRAINT  FOR  AH-1  RUNNING  LANDING 
(SOURCE:  COST  AND  TRAINING  EFFECTIVENESS  ANALYSIS  OF  THE  AH-1  FLIGHT  SIMULATOR) 


r»: 


INITIAL  SIMULATOR  TRAINING  TIME  (MINUTES) 


The  procedure  used  by  the  U.S.  Army  determined  the  most 
cost-effective  means  of  employing  the  flight  simulator  and 
aircraft;  however,  the  question  of  training  effectiveness 
still  remained  unanswered.  To  determine  this  effectiveness, 
aircraft  "checkrides"  were  given  to  both  groups  with  scores 
taken  for  each  of  the  44  tasks  [Ref.  103].  This  author 
conducted  a  hypothesis  test  using  the  average  scores  on  the 
"checkrides"  with  the  following  results: 

Ho  :  PE  "  yC  =  0 

H1  5  “  UC  >  0 

ts  *  2.796 
df  =  66 
PV  =  .0034 

where  E  =  experimental  group 
C  =  control  group. 

Since  the  PV  is  less  than  .01,  we  may  reject  the  null 
hypothesis  that  the  population  checkride  scores  are  equal 
and  accept  the  alternative  hypothesis  that  "checkride" 
scores  for  the  experimental  group  are  greater  than  "check- 
ride"  scores  for  the  control  group. 

E.  THE  ANSER  MODEL 

Presently,  economic  analyses  conducted  in  conjunction 
with  procurement  of  flight  simulators  do  not  evaluate  the 
effects  of  variations  in  training  effectiveness.  The  Air 
Force  has  three  such  models,  which  are  utilized  during  the 
simulator  acquisition  and  testing  cycle,  to  include:  the 
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RCA  Cost  Model,  the  Logistic  Support  Cost  Model,  and  the 
Air  Force  Test  and  Evaluation  Center  (AFTEC)  Cost  of  Owner¬ 
ship  Model.  All  of  these  models  treat  training  effectiveness 
as  fixed.  In  other  words,  the  assumption  is  made  that  the 
training  alternatives  of  interest  are  equally  effective; 
therefore,  simple  cost  comparisons  will  suffice  for  cost- 
effectiveness  analysis  purposes  [Ref.  104]  . 

Cost-effectiveness  analysis  does  not  simply  mean  collect¬ 
ing  cost  and  effectiveness  data.  A  methodology,  or  a  model, 
must  be  defined  that  would  simultaneously  consider  both  cost 
and  effectiveness  inputs,  and  allow  trade-offs  between  these 
inputs  in  order  to  efficiently  evaluate  resource  allocation 
alternatives  [Ref.  105] .  The  cost-effectiveness  model  must 
have  certain  characteristics,  to  include:  (3.)  a  formalized 
framework  so  that  all  alternatives  can  be  compared  in  a  con¬ 
sistent  manner;  (2)  sufficient  flexibility  to  accommodate 
differences  that  exist  between  device  training  applications; 
(3)  support  decisions  between  various  training  alternatives 
on  the  basis  of  measured  or  predicted  cost  and  training 
effectiveness  indices;  (4)  be  sensitive  to  change  with  re¬ 
gards  to  performance,  utilization,  or  capability  of  the 
device;  (5)  include  all  relevant  cost  categories  associated 
with  training;  and  (6)  be  computerized  to  permit  practical 
application  [Ref.  106],  The  characteristics  described  are 
available  within  the  ANSER  model. 
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The  primary  purpose  of  the  ANSER  model  is  to  identify 
the  most  cost-effective  mix  of  training  devices  (including 
the  aircraft,  full-mission  simulators,  cockpit  procedures 
trainers,  part- task  trainers,  media,  and  classroom  instruc¬ 
tion)  for  aircrew  training  on  a  given  weapons  system.  In 
order  to  accomplish  this  purpose,  the  model  requires  input 
data  on  training  requirements  and  device  training  capabili¬ 
ties.  Once  all  mixes  of  devices  that  can  satisfy  the  training 
requirements  are  determined ,  the  model  uses  device  acquisition 
and  operating  costs  to  select  the  most  cost-effective  mixes 
and  compute  the  life  cycle  costs  of  these  sets  of  devices 
[Ref.  107]. 

The  number  of  training  devices  from  which  to  make  a 
choice  is  modeled  as  a  controlled  variable  while  procurement 
costs  and  operations  and  maintenance  costs  are  considered 
fixed  parameters.  State  variables,  such  as  utilization 
rates,  are  dependent  upon  the  controllable  variable.  By 
testing  all  combinations  of  the  integer  values  of  the  con¬ 
trollable  variables,  the  most  cost-effective  combination  of 
devices  can  be  determined  [Ref.  108] . 

Input  data  for  training  requirements  includes  the  number 
of  trciining  tasks,  the  average  number  of  hours  required  in  a 
device  for  a  student  to  accomplish  each  training  component, 
and  the  number  of  students  to  be  trained.  Data  needed  for 
the  training  device  includes  the  ability  of  each  device  to 
satisfy  each  training  component  and  the  maximum  time  each 
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device  can  be  used  for  training  purposes.  To  determine  the 
ability  cf  a  device  to  satisfy  a  training  component,  the 
task  must  be  evaluated  in  terms  of  whether  it  is  transferable 
or  non-transferable.  Transferable  requirements  can  be  accom¬ 
plished  on  a  number  of  different  devices,  while  non-transfer¬ 
able  requirements,  such  as  critical  emergency  procedures 
training,  can  only  be  accomplished  on  a  specific  device. 

This  information  is  very  important  when  trying  to  allocate 
the  time  available  on  each  device  [Ref.  1G9] . 

For  each  task,  devices  are  inserted  into  a  matrix,  in 
order  of  increasing  capability.  Times  are  then  assigned  to 
the  devices  to  meet  the  total  training  requirement.  The 
times  are  assigned  first  to  the  least  capable  device  up  to 
its  maximum  capability.  This  process  is  repeated  for  each 
device  until  the  complete  training  requirement  is  met.  If 
the  requirements  and  capabilities  can  be  successfully  meshed, 
an  effective  alternative  has  been  determined,  therefore 
applicable  cost  data  can  be  calculated  [Ref.  110] .  Figure 
V-4  provides  a  simplified  schematic  for  the  effectiveness 
determ  X  j.*  ation. 

Input  data  for  training  costs  includes  procurement  cost 
of  each  training  device,  operating  and  support  costs  for  each 
device,  the  economic  service  life  for  each  device,  and  appro¬ 
priate  discount  and  inflation  factors.  Figure  V-5  shows  the 
cost  process  associated  with  the  model. 
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Training  Program 


Cost  comparisons  are  based  on  the  average  annual  cost 
for  each  of  the  effective  alternatives.  The  equation  for 


determining  average  annual  cost  is: 


AVERAGE  ANNUAL  COST 


_  TOTAL  PRESENT  VALUE 

SUM' OF  ANNUAL  DISCOUNT  AND/OR 
INFLATION  FACTORS 


This  equates  to  the  normal  present  value  equation  of 
P  =  ^  ,  where  P  =  present  value,  Fn  =  future  value  in 

year  n,  r  =  s'im  of  inflation  and  discount  factors,  and  n  = 
year  or  period  [Ref.  111]. 

Major  cost  elements  used  in  the  model  are. 

Acquisition  Costs 

-Research,  Development,  Training  and  Evaluation 

-Engineering  Development 

-Procurement 

Operation  Costs 

-Operations  Manpower 
-Base  System  Maintenance  Manpower 
-Base  Maintenance  Material 
-Miscellaneous  Personnel  Support 
-Utilities/Fuel 

-Costs  Associated  with  Temporary  Duty 

Base  Operating  Support  Costs 
-Base  Services  Manpower 
-Miscellaneous  Personnel  Support 

Logistics  Support  Costs 

-Depot  Maintenance  Manpower  and  Material 
-Supply  Depot  Manpower  and  Material 
-Second  Destination  Transportation 
-Technical  Order  Maintenance 

Personnel  Support  Costs 

-Recruit  Technical  Training  Manpower 
-Technical  Training  Costs 
-Medical  Manpower 
-M;idical  Material 

-Permanent  Change  of  Station  Costs 
-Miscellaneous  Personnel  Support 


Recurring  Investment  Costs 
-Replenishment  Spares 
-Recurring  Modifications 
-Common  Support  Equipment  [Ref.  112]. 

Readers  of  this  thesis  that  are  interested  in  further  defin¬ 
itions  of  each  cost  element  are  encouraged  to  obtain  AFHRL 
Study  TR-79-39 . 

The  final  output  of  the  model  is  a  readout  which  provides 
information  on  ten  effective  alternatives.  First,  it  lists 
the  number  of  each  device  that  is  needed  to  meet  the  training 
requirements.  Secondly,  it  shows  the  investment  cost,  opera¬ 
tions  and  support  costs,  annual  cost,  and  cumulative  cost 
for  each  year  of  the  economic  service  life.  The  investment 
cost  and  operation  and  support  costs  are  shown  in  constant 
dollars  while  the  annual  and  cumulative  costs  are  shown  in 
inflated/discounted  dollars.  Thirdly,  a  matrix  is  depicted 
outlining  for  each  task  which  device  would  best  meet  the 
training  requirements.  In  other  words,  it  ranks  the  devices 
in  order  of  capability.  Finally,  a  matrix  is  depicted  which 
shows  the  number  of  hours  that  a  student  should  use  each 
training  device  for  each  particular  task. 

Although  the  ANSER  model  offers  sufficient  promise  as 
a  true  cost-effectiveness  model,  it  does  have  limitations. 

The  model  does  not  consider  the  learning  curve;  theory.  That 
is,  the  model  assumes  that  successive  increments  in  device 
use  are  reflected  by  corresponding  increments  in  training 
effectiveness.  A  second  limitation  is  that  the  model  is  not 
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sensitive  to  significant  variations  in  the  input  data  [Ref. 
113J .  Presently  the  Tactical  Air  Warfare  Center  at  Eglin 
AFB,  Florida  is  working  with  the  model  to  correct  these 
limitations.  Their  results  will  be  of  significant  importance 
to  any  department  involved  with  flight  simulator  procurement 
and  utilization. 

F.  THE  COST-EFFECTIVENESS  OF  USING  SIMULATORS 

IN  THE  PILOT  SELECTION  PROCESS 

In  the  past,  using  flight  simulators  as  a  step  in  the 
screening  process  for  officers  to  be  accepted  to  undergrad¬ 
uate  pilot  training  has  not  been  of  significant,  importance 
because  of  the  abundant  supply  of  qualified  applicants  and 
the  relatively  low  training  costs;  however,  with  the  higher 
fuel  costs  and  tne  sophistication  of  current  aircraft  weapons 
systems,  it  is  very  critical  that  the  personnel  that  are 
selected  have  the  highest  probability  for  successful  comple¬ 
tion  of  the  flight  program.  Objective  data  obtained  by  using 
a  flight  simulator  could  provide  early  identification  of 
pilot  candidates  that  are  likely  to  attrite  and  improve  the 
method  by  which  pilots  are  selected  for  different  aircraft 
such  as  fighters,  transport,  or  helicopters. 

The  Royal  Air  Force  has  completed  an  extensive  analysis 
in  using  a  simulator  test  as  a  criterion  for  pilot  selection. 
After  a  four-day  instruction  period  in  basic  instrument  fly¬ 
ing  and  use  and  interpretation  of  radio  navigation  aids,  the 
student  pilots  flew  a  predetermined  route,  to  include  take-off 
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and  climb  to  altitude,  level,  climbing,  and  descending  turns, 
use  of  navigational  aids,  and  approach  to  landing.  The  two 
main  sources  of  data  were  the  instructor* 3  assessment  and 
objective  measures  of  performance.  Correlations  of  .6  to  .8 
were  obtained  between  the  objective  scores  and  the  simulator 
instructor's  assessments.  In  addition,  the  biserial  correla¬ 
tion  of  the  objective  scores  with  the  criterion  of  pass/fail 
in  flying  training  was  approximately  .7  [Ref.  114]. 

Although  the  flight  simulator  in  the  Royal  Air  Force 
analysis  was  shown  to  be  a  good  predictor  of  success  or  fail¬ 
ure  in  flight  training,  there  were  certain  costs  which  accrued 
in  the  process.  The  attrition  rate  for  trainees  entering 
flight  school  decreased,  but  the  introduction  of  the  simulator 
into  the  selection  process  initiated  a  new  attrition  point, 
that  is,  the  trainees  who  failed  the  simulator  test.  Overall, 
this  could  result  in  a  lower  number  of  trainee?  completing 
the  program,  creating  manpower  shortages  for  front-line 
squadrons.  In  order  to  overcome  this  problem,  the  input 
must  be  larger  in  order  to  maintain  a  fixed  output.  In  other 
words,  low  attrition  rates  can  be  achieved  by  using  simulator 
testing;  however,  for  a  fixed  output,  the  cost  savings  assoc¬ 
iated  with  this  reduced  attrition  may  be  offset  by  an  increase 
in  recruiting  and  testing  costs  [Ref.  115] . 

There  are  at  least  two  models  which  have  been  developed 
to  assist  the  decision  maker  in  determining  the  optimal  attri¬ 
tion  rate  for  the  simulator.  These  models  are  the  Recruitment, 
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Selection,  a;.d  Training  (RESET)  coat  model  developed  by 
L.  V.  Bennett  and  H.  C.  Partridge  (1976),  and  the  "CAPER" 
model  developed  by  W.  A.  Souls  of  the  Navy  Personnel  Research 
and  Development  Laboratory.  By  applying  realistic  cost 
values  to  the  model,  it  is  possible  to  calculate:  (1)  the 
cost  savings  resulting  from  lowered  attrition  rates  and 
course  sizes  in  training;  and  (2)  the  increased  costs  of 
selection  resulting  from  larger  intake  numbers  to  meet  a 
fixed  output  [Ref.  116] .  Figure  V-6  shows  the  combination 
of  these  two  costs  as  a  U-shaped  function.  The  lowest  point 
on  the  curve  represents  the  most  cost-effective  attrition 
rate  for  the  simulator  selection  test.  The  x-axis  represents 
a  base  rate  which  is  the  cost  of  selection  and  training  prior 
to  ue  simulator  system,  and  the  Y-axis  shows  marginal  costs 
from  the  base  rate  as  a  function  of  the  attrition  rate.  The 
figure  shows  that  the  maximum  cost  savings  is  at  a  point 
where  the  simulator  attrition  rate  is  40  percent  to  50  per¬ 
cent,  resulting  in  a  cost  savings  of  approximately  300,000 
British  pounds  [Ref.  117]. 

Finally,  the  Royal  Air  Force  analyzed  thoroughly  the 
positioning  of  the  simulator  test  in  the  complete  training 
process.  It  was  determined  that  the  most  cost-effective 
point  to  apply  the  simulator  test  was  after  the  existing 
selection  tests  as  it  was  the  most  expensive  test  to  admin¬ 
ister.  This  allowed  large  numbers  of  the  applicant  popula¬ 
tion  to  be  screened  out  with  lower  costs  validity  rests 
before  using  the  simulator  [Ref.  118]. 
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FIGURE  V-6 

THE  RELATIONSHIP  BETWEEN  ATTRITION  RATE  ON  THE  SIMULATOR  TEST 


Within  the  present  aviation  selection  system  used  fay 
the  Navy,  all  pilot  candidates  are  selected  on  the  basis  of 
performance  in  written  examinations.  The  written  selection 
tests  include  the  Academic  Qualification  Test,  Mechanical 
Comprehension  Test,  Spatial  Apperception  Test,  and  Biograph¬ 
ical  Inventory.  This  battery  of  tests  is  followed  by  an 
extensive  flight  physical  and  a  preflight  school  before 
flight  training.  This  system  has  been  basically  unchanged 
since  World  War  II;  however,  the  Navy  is  presently  evaluating 
a  system  entitled  the  Dynamic  Naval  Aviation  Selection  Test¬ 
ing  and  Evaluation  System  (DYNASTES) .  DYNASTES  is  a  battery 
of  five  tests  which  measures  performance  in  the  following 
areas:  complex  psychoraotor  coordination,  divided  attention, 
selective  attention,  motion  reactivity,  and  vehicular  control. 
Divided  attention  using  the  Integrated  Multi-Task  Psychomotor 
and  Cognitive  Testing  System  (IMPACT)  and  vehicular  control 
using  the  Naval  Automated  Pilot  Aptitude  Measurement  System 
(NAP AMS)  are  completed  on  a  computer  system  consisting  of  a 
.PDP-11/34  Central  Processing  Unit,  a  VT-11  Graphic  Display, 
an  RX02  Duel  Floppy  Disk,  and  an  Audio  Visual  Module. 

The  pass-through  Validation  is  being  conducted  at  the 
present  time  with  400  subjects  undergoing  the  testing  pro¬ 
cedure.  Eight  students  will  receive  two  hours  of  testing 
a  day  for  five  consecutive  days.  This  process  will  be 
repeated  every  week  for  50  weeks.  The  validation  began  in 
October  1980. 
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It  is  estimated  that  the  annual  cost  savings  for  a  5  per¬ 
cent  reduction  in  attrition  (flight  hours  only)  would  be  $3.0 
million.  If  one  considers  that  the  system  has  a  research, 
development,  training,  and  evaluation  cost,  plus  first-year 
implementation  cost  of  $1.3  million,  this  allows  amortization 
within  the  first  year  and  a  resultant  savings  of  $1.7  million. 

The  importance  of  the  DYNASTES  program  is  effectively 
emphasized  when  studying  the  cost  figures  associated  with 
Naval  Air  Training  Command  attrition  rates  for  FY  79  in  Table 
V-3 . 

TABLE  V-3 

NAVAL  AIR  TRAINING  COSTS  ASSOCIATED  WITH  ATTRITION 

(JET  TRAINING) 

(Source:  Chief  of  Naval  Education  and  Training,  Code  N-43A) 


Stage 

Number  of 
Attrites 

Cost  per 
Attrite 

Total 

Primary  Stage 

171 

$  36,453.97 

$  6,233,628.87 

Basic  Stage 

43 

155,238.37 

6,675,249.91 

Advanced  Stage 

28 

407,507.40 

11,410,207.20 

G.  SUMMARY 

TOTAL  COST  - 

$24,319, 085.99 

(Ref.  119] 

Section  V  has  outlined  different  methods  used  by  the 
military  services  to  measure  cost-effectiveness  of  flight 
simulators.  Of  particular  importance  was  the  method  by  which 
microeconomic  theory  could  be  used  to  determine  the  optimum 
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mix  of  time  that  should  be  spent  in  the  airplane  and  simu¬ 
lator  in  order  to  attain  a  fixed  level  of  effectiveness  at 
the  lowest  cost. 

Section  V  also  described  the  results  of  an  analysis  con¬ 
ducted  by  the  Royal  Air  Force  when  using  flight  simulators 
as  a  screening  point  in  its  pilot  selection  process. 

Finally,  Section  V  described  a  program  which  is  currently 
undergoing  evaluation  by  the  U.S.  Navy  to  reduce  the  student 
pilot  attrition  rate  by  having  the  trainee  complete  a  more 
demanding  screening  process  to  include  measuring  his  or  her 
ability  in  operating  a  simplified  flight  simulator. 

Section  VI  will  discuss  this  author's  conclusions  and 
recommendations  concerning  the  training  effectiveness  and 
cost-effectiveness  of  military  flight  simulators. 
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VI.  SUMMARY,  CONCLUSIONS/  AND  RECOMMENDATIONS 


A.  SUMMARY 

The  purpose  of  this  thesis  was  to  conduct  research  in 
the  area  of  visual  flight  simulation  and  determine  whether 
or  not  systems  that  exist  provide  both  training  and  cost 
effectiveness . 

Section  II  provided  an  overview  of  flight  simulation  to 
include  the  historical  development  of  flight  simulation,  a 
description  of  visual  flight  simulation  methodology  such  as 
model  boards,  computer  generated  imagery,  and  computer  ani¬ 
mated  photographic  terrain  view,  and  a  description  of  military 
flight  simulators  that  are  presently  being  used  in  the  train¬ 
ing  syllabi  for  Air  Force  and  Navy  aircraft. 

Section  III  examined  the  use  of  flight  simulators  within 
the  commercial  aviation  industry.  Methods  by  which  the  major 
airline  companies  used  the  systems  to  provide  training  effec¬ 
tiveness  were  outlined,  as  well  as  cost  savings  realized  by 
major  carriers  such  as  American  and  Delta  Airlines.  A  major 
point  disclosed  in  Section  III  was  the  improvement  in  standard¬ 
ization  of  training  and  economies  of  scale  gained  by  the 
airlines  centralizing  their  respective  training  centers. 

Section  IV  evaluated  the  effectiveness  of  simulator 
systems  in  training  military  pilots.  It  outlined  factors  such 
as  design  of  simulator  systems,  training  programs,  personnel, 


and  expectations,  discussing  how  each  factor  could  effect 
the  implementation  of  simulator  systems  into  the  military 
training  concept.  Hypothesis  testing  was  completed  on  data 
that  were  gathered  from  air-to-ground  training,  air-to-air 
training,  and  night  carrier  landing  training.  All  studies 
examined  supported  the  fact  that  flight  simulators  did  pro¬ 
vide  effective  training. 

Finally,  Section  V  discussed  the  cost-effectiveness  of 
military  flight  simulators.  A  cost-benefit  analysis  used  by 
the  Navy  in  the  procurement  process  was  applied  to  Air  Force 
cost  data  with  the  result  being  that  investment  costs  were 
amortized  in  slightly  less  than  three  years.  Microeconomic 
theory  was  applied  in  determining  the  optimum  mix  of  training 
hours  between  the  airplane  and  simulator  that  could  be  utilized 
in  meeting  a  predetermined  fixed  effectiveness  at  the  minimum 
cost.  A  CTEA  analysis  conducted  by  the  Army  for  the  AH-1 
"Cobra"  helicopter  was  outlined  using  task  training  analysis, 
cost  analysis,  and  task  priority.  The  model  developed  by 
Analytic  Services  (ANSER  MODEL)  to  determine  cost-effectiveness 
of  aircrew  training  devices  was  described  with  the  important 
point  being  made  that  aircrew  training  devices  included  the 
aircraft,  simulators,  media,  and  classroom  instruction.  The 
model  used  input  data  of  training  requirements,  device  trainxng 
capabilities,  and  pertinent  costs  to  determine  the  optimum  mix 
of  training  devices  to  meet  the  training  requirements.  The 
model  would  provide  ten  effective  alternatives  with  their 
respective  life-cycle  costs. 
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The  .last  part  of  Section  V  explained  an  analysis  con¬ 
ducted  by  the  Royal  Air  Force  to  use  flight  simulators  in 
the  process  of  selecting  personnel  for  pilot  training,  and 
a  procedure,  DYNASTES,  which  is  currently  being  validated  by 
the  Navy  in  their  pilot  selection  process.  A  figure  was  used 
to  depict  the  possible  cost-tradeoffs  of  using  a  simulator  in 
the  selection  process,  pointing  out  the  importance  of  an 
accurate  attrition  rate  from  the  simulator  test  in  order  to 
preclude  the  possibility  of  having  the  input  costs,  such  as 
recruitment  and  selection,  be  greater  than  the  cost  savings 
of  training  fewer  pilots  in  the  aircraft. 

B.  CONCLUSIONS 

1.  Flight  Simulators  Improve  Training  Effectiveness 
This  thesis  has  determined  that  flight  simulators 
can  improve  the  training  effectiveness  of  military  pilots. 

The  major  problem  is  convincing  the  pilot  that  the  system 
will  not  be  used  as  a  substitute  for  actual  aircraft  time. 

Even  though  state-of-the-art  technology  provides  accurate 
"realism,"  the  typical  military  pilot  needs  that  valuable 
time  in  the  aircraft  to  perfect  his  flying  techniques,  thereby 
increasing  the  probability  of  success  in  an  engagement  with 
the  enemy.  There  mu3t  be  a  minimum  monthly  flight  hour 
requirement  set  for  military  pilots  so  that  his  or  her  flying 
akility  can  be  maintained.  Even  though  the  cost  associated 
with  this  minimum  flight  time  can  be  quite  significant,  by 
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using  flight  simulation  in  the  training  process  to  practice 
a  particular  maneuver  or  whatever  Lark  is  required,  the  pilot 
is  able  to  use  the  actual  aircraft  time  more  effectively. 

2.  Flight  Simulators  Can  Increase  Long-Run  Cost  Savings 
It  is  this  author's  opinion  that  short-run  cost  sav¬ 
ings  associated  with  flight  time  substitution  has  been  the 
key  factor  in  determining  whether  or  not  funds  are  authorized 
for  flight  simulator  procurement.  Using  the  simulator  to 
practice  air-to-ground  ordnance  delivery  or  firing  air-to-air 
missiles  can  result  in  a  long-run  cost  savings  because  the 
pilots  would  not  require  as  much  practice  ordnance  in  order 

to  attain  a  certain  level  of  effectiveness.  More  importantly, 
there  can  be  a  greater  cost  savings  associated  with  accident 
reduction  as  a  result  of  the  simulator/aircraft  training 
combination  because  this  process  has  been  shown  to  produce 
better  qualified  aviators. 

3 .  The  Cost-Effectiveness  of  Flight  Simulation 
Can  Be  Improved 

Determining  the  least  cost  mix  of  aircraft  and  simu¬ 
lator  time  in  order  to  attain  a  predetermined  effectiveness 
can  significantly  improve  the  cost-effectiveness  of  the 
system.  Objective  data  can  develop  both  the  effectiveness 
isoquants  and  the  cost  constraints,  allowing  the  user  of  the 
system  to  apply  the  microeconomic  theory  that  was  outlined 
in  this  thesis. 
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4.  All  Levels  of  Command  Must  Understand  and 


This  conclusion  is  of  primary  importance,  especially 
for  the  fleet  user.  Nothing  is  more  detrimental  to  the 
training  process  than  to  have  an  instructor  not  understand 
the  capabilities  and/or  limitations  of  the  system  or  not 
believe  the  system  can  improve  the  trainee's  ability.  It  is 
imperative  that  flight  simulation  be  eagerly  accepted  as  an 
integral  part  of  the  training  process;  otherwise,  utilization 
rates  will  be  low  and  costs  will  be  high. 

C.  RECOMMENDATIONS 

Since  the  Marine  Corps  is  on  the  threshold  of  procuring 
sophisticated  flight  simulators  for  both  the  F/A-18  and  the 
AV-8B,  it  is  recommended  that  careful  study  be  given  to  the 
following  areas  of  this  thesis: 

1.  The  factors  influencing  training  effectiveness  out¬ 
lined  in  Section  IV.  It  is  imperative  that  training  syllabi 
be  developed  that  allow  an  interface  between  the  sinmlator 
and  the  aircraft-  and  that  the  personnel  operating  the  console 
of  the  simulator  be  professionally  trained  so  that  the  systems 
can  function  at  maximum  capability. 

2.  The  hypothesis  testing  completed  in  Section  IV  which 
supported  the  fact  that  flight  simulators  can  provide  excellent 
training  in  different  flying  environments. 

3.  The  microeconomic  theory  in  Section  V  which  could 
be  applied  to  pilots  using  the  F/A-18  and  AV-8B,  allowing 
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effectiveness  isoquants  to  be  developed  for  all  tasks  that 
could  be  learned  in  either  the  simulator  or  the  aircraft. 

4.  The  ANSER  MODEL,  outlined  in  Section  V,  to  determine 
whether  or  not  it  can  be  utilized  by  the  Marine  Corps  in 
future  weapons  systems  procurement. 
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APPENDIX  A 


SIMULATOR  AND  VISUAL  SYSTEM  REQUIREMENTS 


a.  Discussion.  For  the  convenience  of  the  reader,  the 
simulator  and  visual  system  requirements  of  Appendix  H  to 
Part  121  have  been  included  in  this  appendix.  Clarification 
has  been  included  for  some  requirements.  The  preamble  to 
the  Advanced  Simulation  Rule  contains  additional  guidance  re¬ 
garding  these  requirements,, 

b.  Simulator  Requirements  -  General. 

(1)  The  cockpit  should  represent  a  full-sc?ile  mockup 
of  the  aircraft  simulated.  Where  movement  of  controls  and 
switches  is  involved,  the  direction  of  movement  should  be 
identical  to  that  in  the  applicant's  aircraft. 

(2)  Circuit  breakers  that  affect  procedures  and 
functions  resulting  in  observable  cockpit  indications  should 
be  functionally  accurate. 

(3)  The  effect  of  aerodynamic  changes  for  various 
combinations  of  drag  and  thrust  normally  encountered  in  flight 
should  correspond  to  actual  flight  conditions.  The  effect  of 
change  in  aircraft  attitude,  thrust,  drag,  altitude,  tempera¬ 
ture,  gross  weight,  center  of  gravity  location,  and  configur¬ 
ation  should  be  included. 

(4)  All  relevant  instrument  indications  involved  in 
the  simulation  of  the  applicable  aircraft  should  be  entirely 
automatic  in  response  to  control  movement  by  a  crewmember. 

(5)  The  rate  of  change  of  simulator  instrument  read¬ 
ings  and  of  control  forces  should  correspond  to  the  rate  of 
change  which  would  occur  on  the  applicable  aircraft  under 
actual  flight  conditions  for  any  given  change  in  forces 
applied  to  the  controls,  in  the  applied  power,  or  in  aircraft 
configurations . 

(6)  Control  forces  and  degree  of  actuation  control 
travel  should  correspond  to  that  which  would  occur  in  the 
aircraft  under  actual  flight  conditions. 

(7)  Communications  and  navigation  equipment  should 
correspond  to  that  installed  in  the  applicant's  aircraft  and 
should  operate  within  the  tolerances  prescribed  for  the  actual 
airborne  equipment.  Long  range  navigation  systems  should  be 
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installed  but  need  not  be  operative  unless  required  by  Part 
121,  Appendix  H. 

(8)  In  addition  to  the  flight  crewmember  stations, 
there  should  be  two  suitable  seat  accommodations  for  the 
Instructor/Check  Airman  and  FAA  Inspector.  Operators  who 
have  the  Check  Airman/Instructor  occupy  a  flightcrew  position 
seat  need  only  provide  one  additional  observer  seat.  These 
seats  should  provide  adequate  vision  to  the  pilot's  panel 
and  forward  windows  in  visual  system  models.  Observer  seats 
need  not  represent  the  aircraft  seats. 

(9)  Simulator  systems  should  simulate  the  applicable 
aircraft  system  operation,  both  on  the  ground  and  in  flight. 
Major  systems  should  be  operative  to  the  extent  that  normal 
operating  procedures,  and  abnormal  and  emergency  procedures 
included  in  the  operator's  programs  can  be  accomplished. 

(10)  An  Instructor  Control  Console  should  be  installed 
to  enable  the  Instructor/Check  Airman  or  FAA  Inspector  (when 
applicable)  to  control  the  visual  attachment  (if  installed) 
and  insert  abnormal  or  emergency  conditions  into  the  aircraft 
systems. 


c*  Visual  Requirements  -  General. 

(1)  The  visual  scene  should  accurately  portray  the 
environment  equivalent  to  that  which  the  pilot  observes  on 
the  related  simulator  cockpit  instrument  display  resulting 
from  the  manipulation  of  the  controls  and  the  effects  of 
varying  wind  conditions. 

(2)  The  visual  display  may  be  either  a  monoview  or 
duoview  display.  If  a  monoview  display  is  used,  it  should  be 
capable  of  transfer  of  display  at  either  pilot  station. 

(3)  The  scene  should  comprise  the  airfield,  surround¬ 
ing  area,  airport  ramp  and  taxiway. 

(4)  Representations  of  buildings  or  other  outstanding 
features  should  be  suitably  detailed  to  produce  a  realistic 
effect  on  picture  presentation. 

(5)  Functional  airfield  and  approach  lighting  should 
be  representative  of  the  runway  depicted  with  intensity 
controls  to  vary  degree  of  lightness.  Approach  and  runway, 
and  lighting  intensities  should  be  independently  variable. 
Realistic  colors  for  approach,  and  runway  lighting  are  re¬ 
quired.  Computer-generated  image  (CGI)  systems  should  have 
the  capability  of  portraying  runway  texture  or  surface. 
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(6)  The  aircraft  landing  lights  should  be  operational. 

(7)  The  optical  system  for  Phase  I  and  less  sophis¬ 
ticated  simulators  should  be  capable  of  providing  at  least  a 
45°  field  of  vision.  Focus  should  be  automatic  in  order  to 
keep  at  optimum  that  part  of  the  picture  which  is  signifi¬ 
cant  to  the  pilot.  A  minimum  of  75°  horizontally  and  30° 
vertically  is  required  for  Phase  II  and  III  visual  systems. 

(8)  An  instructor's  control  should  be  provided  to 
allow  control  of  all  aspects  of  the  visual  system;  i.e., 
cloudbase,  visibility  in  miles  and  feet,  airport  selection, 
environmental  lighting  controls,  VASI,  etc. 

(9)  Visual  systems  approved  for  instrument  takeoffs 
and/or  instrument  approach  procedures  should  have  a  means  of 
reducing  visibility  to  reasonably  simulate  the  appropriate 
weather  conditions . 

(10)  Operators  possessing  visual  systems  that  do  not 
meet  all  the  requirements  contained  in  this  paragraph  and 
have  received  prior  approval  will  have  "grandfather  rights." 
These  systems  will  be  eligible  for  continued  approval  for  all 
maneuvers  originally  approved  provided  they  are  maintained 
to  the  level  of  acceptability  demonstrated  at  original 
approval.  The  "grandfather  rights"  apply  only  to  the  original 
operator  and  are  not  transferable. 

d.  Simulator  Requirements  -  Phase  I. 

(1)  Aerodynamic  programming  to  include: 

(a)  Ground  effect — for  example,  roundout,  flare, 
and  touchdown.  This  requires  data  on  lift,  drag,  and  pitching 
moment  in  ground  effect. 

(b)  Ground  reaction— reaction  of  the  airplane 
upon  contact  with  the  runway  during  landing  to  include  strut 
deflections,  tire  friction,  and  side  forces. 

(c)  Ground  handling  characteristics — steering 
inputs  to  include  crosswind,  braking,  thrust  reversing, 
deceleration,  and  turning  radius. 

(2)  Minimum  of  3-axis  freedom  of  motion  systems. 

(3)  Phase  I  landing  maneuver  test  guide  to  verify 
simulator  data  with  actual  airplane  flight  test  data,  and 
provide  simulator  performance  tests  for  initial  approval. 

(4)  Multichannel  recorders  capable  of  recording 
Phase  I  performance  tests. 
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e.  Visual  Requirements  -  Phase  I. 

Cl)  Visual  system  compatibility  with  aerodynamic 
programming. 

(2)  Visual  system  response  time  from  pilot  control 
input  to  visual  system  output  shall  not  exceed  300  milli¬ 
seconds  more  than  the  movement  of  the  airplane  to  a  similar 
input.  Visual  system  response  time  is  defined  as  the  comple¬ 
tion  of  the  visual  display  scan  of  the  first  video  field 
containing  different  information  resulting  from  an  abrupt 
control  input. 

(3)  A  means  of  recording  the  visual  response  time 
for  comparison  with  airplane  data. 

(4)  Visual  cues  to  assess  sink  rate  and  depth  per¬ 
ception  during  landings. 

(5)  Visual  scene  to  instrument  correlation  to 
preclude  perceptible  lags. 

f .  Simulator  Requirements  -  Phase  II. 

(1)  Representative  crosswind  and  three-dimensional 
windshear  dynamics  based  on  airplane  related  data. 

(2)  Representative  stopping  and  directional  control 
forces  for  at  least  the  following  runway  conditions  based  on 
airplane  related  data: 


(a) 

Dry, 

(b) 

Wet. 

(c) 

Icy. 

(d) 

Patchy 

wet. 

(e) 

Patchy 

icy. 

(f) 

Wet  on 

rubber  residue  in  touchdown  zone 

(3)  Representative  brake  and  tire  failure  dynamics 
(including  antiskid)  and  decreased  brake  efficiency  due  to 
high  brake  temperatures  based  on  airplane  related  data. 

These  representations  should  be  realistic  enough  to  cause 
pilot  identification  of  the  problem  and  implementation  of 
appropriate  procedures.  Simulator  pitch,  side  loading  and 
directional  control  characteristics  should  be  representative 
of  the  aircraft. 
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(4)  A  motion  system  which  provides  motion  cues  equal 
to  or  better  than  those  provided  by  a  six-axis  freedom  of 
motion  system. 

(5)  Operational  principal  navigation  systems,  includ¬ 
ing  electronic  flight  instrument  systems,  INS,  and  OMEGA,  if 
applicable.  This  requirement  is  to  enhance  LOFT;  therefore, 
if  the  operator’s  route  structure  requires  dual  long  range 
navigation  systems  on  board  its  aircraft  (i.e.,  Omega,  INS, 
Doppler)  a  sufficient  number  of  simulators,  but  in  no  case 
less  than  one  simulator,  should  be  equipped  with  the  appro¬ 
priate  long-range  navigation  system  utilized. 

(6)  Means  for  quickly  and  effectively  testing  simu¬ 
lator  programming  and  hardware.  This  could  include  an  auto¬ 
mated  system  which  could  be  used  for  conducting  at  least  a 
portion  of  the  tests  in  the  ATG. 

(7)  Expanded  simulator  computer  capacity,  accuracy, 
resolution,  and  dynamic  response  to  meet  Phase  II  demands. 
Resolution  equivalent  to  that  of  at  least  a  32-bit  word 
length  computer  is  required  for  critical  aerodynamic  programs. 

(8)  Timely  permanent  update,  of  simulator  hardware 
and  programming  subsequent  to  airplane  modification. 

(9)  Sound  of  precipation  and  significant  airplane 
noises  perceptible  to  the  pilot  during  normal  operations  and 
the  sound  of  a  crash  when  the  simulator  is  landed  in  excess 
of  landing  gear  limitations.  Significant  airplane  noises 
should  include  noises  such  an  engine  noise,  flap,  gear  and 
spoiler  extension  and  retraction  and  thrust  reversal  to  a 
comparable  level  as  that  found  in  the  aircraft. 

(10)  Aircraft  control  feel  dynamics  shall  duplicate 
the  airplane  simulated.  This  shall  be  determined  by  compar¬ 
ing  a  recording  of  the  control  feel  dynamics  of  the  simulator 
to  airplane  measurements  in  the  takeoff,  cruise,  and  landing 
configuration.  Airplane  measurements  may  be  obtained  on  the 
ground  if  proper  pitot  static  inputs  are  provided  to  repre¬ 
sent  airspeeds  typical  of  those  encountered  on  takeoff, 
cruise  and  landing.  This  should  provide  control  feel 
measurements  comparable  to  those  encountered  in  flight. 

(11)  Relative  responses  of  the  motion  system,  visual 
system,  and  cockpit  instruments  shall  be  coupled  closely  to 
provide  integrated  sensory  cues.  These  systems  shall  respond 
to  abrupt  pitch,  roll,  and  yaw  inputs  at  the  pilot's  position 
within  150  milliseconds  of  the  time,  but  not  before  the  time, 
when  the  airplane  would  respond  under  the  same  conditions. 
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Visual  scene  changes  from  steady  state  disturbance  shall  not 
occur  before  the  resultant  motion  onset  but  within  the  system 
dynamic  response  tolerance  of  150  milliseconds.  The  test  to 
determine  compliance  with  these  requirements  shall  include 
simultaneously  recording  the  analog  output  from  the  pilot’s 
control  column  and  rudders,  the  output  from  an  accelerometer 
attached  to  the  motion  system  platform  located  at  an  acceptable 
location  near  the  pilots'  seats,  the  output  signal  to  the 
visual  system  display  (including  visual  system  analog  delays) , 
and  the  output  signal  to  the  pilot's  attitude  indicator  or  an 
equivalent  test  approved  by  the  Administrator.  The  test 
results  in  a  comparison  of  a  recording  of  the  simulator's 
response  to  actual  airplane  response  data  in  the  takeoff, 
cruise,  and  landing  configuration. 

g.  Visual  Requirements  -  Phase  II. 

(1)  Dusk  and  night  visual  scenes  with  at  least  three 
specific  airport  representations,  including  a  capability  of 
at  least  10  levels  of  occulting,  general  terrain  character¬ 
istics,  and  significant  landmarks.  It  is  not  necessary  for 
each  airport  scene  to  contain  10  levels  of  occulting  but 
there  should  be  a  means  of  demonstrating  that  the  visual 
system  has  that  capability. 

(2)  Radio  navigation  aids  properly  oriented  to  the 
airport  runway  layout. 

(3)  Test  procedures  to  quickly  confirm  visual  system 
color,  RVR,  focus,  intensity,  level  horizon,  and  attitude  as 
compared  to  tha  simulator  attitude  indicator. 

(4)  For  the  approach  and  landing  phase  of  flight,  at 
and  below  an  altitude  of  2,000  feet  height  above  the  airport 
(HAA)  and  within  a  radius  of  10  miles  from  the  airport, 
weather  representations  including  the  following: 

(a)  Variable  cloud  density. 

(b)  Partial  obscuration  of  ground  scenes;  that 
is,  the  effect  of  a  scattered  to  broken  cloud  deck. 

(c)  Gradual  break  out. 

(d)  Patchy  fog. 

(e)  The  effect  of  fog  on  airport  lighting. 

(f)  Category  II  and  III  weather  conditions. 

These  representations  are  required  only  if  the  operator  is 
authorized  to  operate  under  Category  II  or  III  conditions. 
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(5)  Continuous  minimum  visual  field  of  view  of  75° 
horizontal  and  30°  vertical  per  pilot  seat.  Visual  gaps 
shall  occur  only  as  they  would  in  the  airplane  simulated  or 
as  required  by  visual  system  hardware.  Both  pilot  seat 
visual  systems  shall  be  able  to  be  operated  simultaneously. 

(6)  Capability  to  present  ground  and  air  hazards 
such  as  another  airplane  c  .ossing  the  active  runway  or  con¬ 
verging  airborne  traffic. 

h.  Simulator  Requirements  -  Phase  III. 

(1)  Characteristic  buffet  motions  that  result  from 
operation  of  the  airplane  (for  example,  high-speed  buffet, 
extended  landing  gear,  flaps,  nose-wheel  scuffing,  stall) 
which  can  be  sensed  at  the  flight  deck.  The  simulator  must 
be  programmed  and  instrumented  in  such  a  manner  that  the 
characteristic  buffet  modes  can  be  measured  and  compared  to 
airplane  data.  Airplane  data  are  also  required  to  define 
flight  deck  motions  when  the  airplane  is  subjected  to  atmos¬ 
pheric  disturbances  such  as  rough  air  and  cobblestone  turbulence. 
General  purpose  disturbance  models  that  approximate  demonstrable 
flight  test  data  are  acceptable. 

(2)  Aerodynamic  modeling  for  aircraft  for  which  an 
original  type  certificate  is  issued  after  June  1,  1980, 
including  low-altitude,  level-flight  ground  effect,  mach 
effect  at  high  altitude,  effects  of  airframe  icing,  normal 
and  reverse  dynamic  thrust  effect  on  control  surfaces,  aero- 
elastic  representations,  and  representations  of  nonlinearities 
due  to  side  slip  based  on  airplane  flight  test  data  provided 
by  the  manufacturer. 

(3)  Realistic  amplitude  and  frequency  of  cockpit 
noises  and  sounds,  including  precipitation  static  and  engine 
and  airframe  sounds.  The  sounds  shall  be  coordinated  with 
the  weather  representations  required  in  Phase  III  visual 
requirement  No.  3. 

(4)  Self- testing  for  simulator  hardware  and  program¬ 
ming  to  determine  compliance  with  Phase  I,  II,  and  III  simu¬ 
lator  requirements. 

(5)  Diagnostic  analysis  printout  of  simulator  mal¬ 
functions  sufficient  to  determine  MEL  compliance.  These 
printouts  shall  be  retained  by  the  operator  between  recurring 
FAA  simulator  evaluations  as  part  of  the  daily  discrepancy 
log  required  under  121.407(a)(5). 
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i.  Visual  Requirements  -  Phase  III. 


(1)  Daylight,  dusk,  and  night  visual  scenes  with 
sufficient  scene  content  to  recognize  a  specific  airport, 
the  terrain,  and  major  landmarks  around  that  airport  and  to 
successfully  accomplish  a  visual  landing.  The  daylight  visual 
scene  must  be  part  of  a  total  daylight  cockpit  environment 
which  at  least  represents  the  amount  of  light  in  the  cockpit 
on  an  overcast  day.  For  the  purpose  of  this  rule,  daylight 
visual  system  is  defined  as  a  visual  system  capable  of  pro¬ 
ducing,  as  a  minimum,  full  color  presentations,  scene  content 
comparable  in  detail  to  that  produced  by  4,000  edges  or  1,000 
surfaces  for  daylight  and  4,000  light  points  for  night  and 
dusk  scenes,  6-foot  lamberts  of  light  at  the  pilot's  eye 
(highlight  brightness) ,  3-arc  minutes  resolution  for  the 
field  of  view  at  the  pilot's  eye,  and  a  display  which  is  free 
of  apparent  quantization  and  other  distracting  visual  effects 
while  the  simulator  is  in  motion.  The  simulation  of  cockpit 
ambient  lighting  shall  be  dynamically  consistent  with  the 
visual  scene  displayed.  For  daylight  scenes,  such  ambient 
lighting  shall  neither  "washout"  the  displayed  visual  scene 
nor  fall  below  5-foot  lamberts  of  light  as  reflected  from  an 
approach  plate  at  knee  height  at  the  pilot's  station  and/or 
2-foot  lamberts  of  light  as  reflected  from  the  pilot's  face. 

(2)  Visual  scenes  portraying  representative  physical 
relationships  which  are  known  to  cause  landing  illusions  in 
some  pilots,  including  short  runway,  landing  over  water, 
runway  gradient,  visual  topographic  features,  and  rising 
terrain. 


(3)  Special  weather  representations  which  include  the 
sound,  visual,  and  motion  effects  of  entering  light,  medium, 
and  heavy  precipitation  near  a  thunderstorm  on  takeoff, 
approach,  and  landing..  at  and  below  an  altitude  of  2,000  feet 
HAA  and  within  a  radius  of  10  miles  from  the  airport. 

(4)  Phase  II  visual  requirements  in  daylight  as  well 
as  dusk  and  night  representations. 

(5)  Wet  and,  if  appropriate  for  the  operator,  snow- 
covered  runway  representations,  including  runway  lighting 
effects. 


(6)  Realistic  color  and  directionality  of  airport 
lighting. 

(7)  Weather  radar  presentations  in  aircraft  where 
radar  information  is  presented  on  the  pilot's  navigation 
instruments. 
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11.  Dr.  L.  E.  Ryan 

Naval  Training  Equipment  Center,  Code  N-82 
Orlando,  Florida  32813 


12.  Delta  Airlines,  Incorporated  1 

Attn:  Captain  W.  R.  Jones 

Continental  Colony  Parkway 
Atlanta,  Georgia  30331 

13.  American  Airlines  Flight  Academy  1 

Attn:  Mr.  John  Hancock 

American  Airlines  Plaza 
Fort  Worth,  Texas  76125 

14.  Air  Force  Human  Resources  Laboratory  1 

Attn:  Cdr.  Larry  Rasmussen,  USN 

Williams  AFB,  Arizona  85224 

15.  Maj.  J.  L.  Reeve,  USMC  1 

1st  Marine  Aircraft  Wing 

FPO  San  Francisco,  California  96602 

16.  Maj.  John  Lizzo,  USMC  1 

VMAT-203,  MAG-32 

MCAS  Cherry  Point,  North  Carolina  28533 

17.  Dr.  Jesse  Orlansky  1 


Science  and  Technology  Division 
Institute  for  Defense  Analysis 
400  Army- Navy  Drive 
Arlington,  Virginia  22202 

18 .  Seville  Research  Corporation  1 

Attn:  Dr.  Stephen  R.  Osborne 

2121  S.  Mill  Avenue,  Suite  202 
Tempe,  Arizona  85282 

19.  Maj.  George  B.  Mayer,  Jr.  3 

2226  Fox  Horn  Road 

New  Bern,  North  Carolina  28560 

20.  Cdr.  W,  F.  Moroney,  Code  55  Mp  i 

Operations  Research  Department 

Naval  Postgraduate  School 
Monterey,  CA  93940 
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